Remote update programming idiom accelerator with allocated processor resources

ABSTRACT

A data processing system comprises at least one processing unit, a virtualization layer, and a remote update programming idiom accelerator. The remote update programming idiom accelerator is configured to receive a complex remote update programming idiom from a remote node. Responsive to a determination that the sequence of instructions in the complex remote update programming idiom is longer than a dedicated processor threshold, the remote update programming idiom accelerator is configured to request a processing unit from the virtualization layer in the data processing system, and receive an allocation of a processing unit from the virtualization layer. The allocated processing unit is configured to read the data from the storage location local to the data processing system, execute the sequence of instructions to perform the update operation on the data to form result data, and write the result data to the storage location local to the data processing system.

This invention was made with United States Government support underAgreement No. HR0011-07-9-0002 awarded by DARPA. The Government hascertain rights in the invention.

BACKGROUND

The present application relates generally to an improved data processingsystem and method. More specifically, the present application isdirected to a mechanism to wake a sleeping thread based on anasynchronous event.

Multithreading is multitasking within a single program. Multithreadingallows multiple streams of execution to take place concurrently withinthe same program, each stream processing a different transaction ormessage. In order for a multithreaded program to achieve trueperformance gains, it must be run in a multitasking or multiprocessingenvironment, which allows multiple operations to take place.

Certain types of applications lend themselves to multithreading. Forexample, in an order processing system, each order can be enteredindependently of the other orders. In an image editing program, acalculation-intensive filter can be performed on one image, while theuser works on another. Multithreading is also used to createsynchronized audio and video applications.

In addition, a symmetric multiprocessing (SMP) operating system usesmultithreading to allow multiple CPUs to be controlled at the same time.An SMP computing system is a multiprocessing architecture in whichmultiple central processing units (CPUs) share the same memory. SMPspeeds up whatever processes can be overlapped. For example, in adesktop computer, SMP may speed up the running of multiple applicationssimultaneously. If an application is multithreaded, which allows forconcurrent operations within the application itself, then SMP mayimprove the performance of that single application.

If a process, or thread, is waiting for an event, then the process goesto sleep. A process is said to be “sleeping,” if the process is in aninactive state. The thread remains in memory, but is not queued forprocessing until an event occurs. Typically, this event is detected whenthere is a change to a value at a particular address or when there is aninterrupt.

As an example of the latter, a processor may be executing a firstthread, which goes to sleep. The processor may then begin executing asecond thread. When an interrupt occurs, indicating that an event forwhich the first thread was waiting, the processor may then stop runningthe second thread and “wake” the first thread. However, in order toreceive the interrupt, the processor must perform interrupt eventhandling, which is highly software intensive. An interrupt handler hasmultiple levels, typically including a first level interrupt handler(FLIH) and a second level interrupt handler (SLIH); therefore, interrupthandling may be time-consuming.

In the former case, the processor may simply allow the first thread toperiodically poll a memory location to determine whether a particularevent occurs. The first thread performs a get instruction and a compareinstruction (GET&CMP) to determine whether a value at a given address ischanged to an expected value. When one considers that a computing systemmay be running thousands of threads, many of which are waiting for anevent at any given time, there are many wasted processor cycles spentpolling memory locations when an expected event has not occurred.

SUMMARY

In one illustrative embodiment, a method is provided in a dataprocessing system for performing a remote update. The method comprisesreceiving, at a remote update programming idiom accelerator within thedata processing system, a complex remote update programming idiom from aremote node. The complex remote update programming idiom includes a readoperation for reading data from a storage location local to the dataprocessing system, at least one update operation for performing anoperation on the data to form result data, and a write operation forwriting the result data to the storage location local to the dataprocessing system. The method further comprises determining whether thesequence of instruction is longer than a dedicated processor thresholdand responsive to a determination that the sequence of instructions islonger than the dedicated processor threshold, requesting processingresources from a virtualization layer in the data processing system. Themethod further comprises receiving an allocation of processing resourcesfrom the virtualization layer, reading, by the allocated processingresources, the data from the storage location local to the dataprocessing system, executing, by the allocated processing resources, thesequence of instructions to perform the update operation on the data toform the result data, writing, by the processing resources, the resultdata to the storage location local to the data processing system, andreturning a completion notification from the remote update programmingidiom accelerator to the remote node informing the remote node thatprocessing of the complex remote update programming idiom has completedon the data processing system.

In another illustrative embodiment, a data processing system comprisesat least one processing unit, a virtualization layer, and a remoteupdate programming idiom accelerator. The remote update programmingidiom accelerator is configured to receive a complex remote updateprogramming idiom from a remote node. The complex remote updateprogramming idiom includes a read operation for reading data from astorage location local to the data processing system, at least oneupdate operation for performing an operation on the data to form resultdata, and a write operation for writing the result data to the storagelocation local to the data processing system. The remote updateprogramming idiom accelerator is configured to determine whether thesequence of instruction is longer than a dedicated processor threshold,responsive to a determination that the sequence of instructions islonger than the dedicated processor threshold, request a processing unitfrom the virtualization layer in the data processing system, and receivean allocation of a processing unit from the virtualization layer. Theallocated processing unit is configured to read the data from thestorage location local to the data processing system, execute thesequence of instructions to perform the update operation on the data toform result data, and write the result data to the storage locationlocal to the data processing system. The remote update programming idiomaccelerator is configured to return a completion notification from theremote update programming idiom accelerator to the remote node informingthe remote node that processing of the complex remote update programmingidiom has completed on the data processing system.

These and other features and advantages of the present invention will bedescribed in, or will become apparent to those of ordinary skill in theart in view of, the following detailed description of the exemplaryembodiments of the present invention.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The invention, as well as a preferred mode of use and further objectivesand advantages thereof, will best be understood by reference to thefollowing detailed description of illustrative embodiments when read inconjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram of an exemplary data processing system inwhich aspects of the illustrative embodiments may be implemented;

FIG. 2 is a block diagram of a wake-and-go mechanism in a dataprocessing system in accordance with an illustrative embodiment;

FIG. 3 is a block diagram of a wake-and-go mechanism with a hardwareprivate array in accordance with an illustrative embodiment;

FIGS. 4A and 4B are block diagrams illustrating operation of awake-and-go mechanism with specialized processor instructions inaccordance with an illustrative embodiment;

FIGS. 5A and 5B are block diagrams illustrating operation of awake-and-go mechanism with a specialized operating system call inaccordance with an illustrative embodiment;

FIG. 6 is a block diagram illustrating operation of a wake-and-gomechanism with a background sleeper thread in accordance with anillustrative embodiment;

FIGS. 7A and 7B are flowcharts illustrating operation of a wake-and-gomechanism in accordance with the illustrative embodiments;

FIGS. 8A and 8B are flowcharts illustrating operation of a wake-and-gomechanism with prioritization of threads in accordance with theillustrative embodiments;

FIGS. 9A and 9B are flowcharts illustrating operation of a wake-and-gomechanism with dynamic allocation in a hardware private array inaccordance with the illustrative embodiments;

FIG. 10 is a block diagram of a hardware wake-and-go mechanism in a dataprocessing system in accordance with an illustrative embodiment;

FIGS. 11A and 11B illustrate a series of instructions that are aprogramming idiom for wake-and-go in accordance with an illustrativeembodiment;

FIGS. 12A and 12B are block diagrams illustrating operation of ahardware wake-and-go mechanism in accordance with an illustrativeembodiment;

FIGS. 13A and 13B are flowcharts illustrating operation of a hardwarewake-and-go mechanism in accordance with the illustrative embodiments;

FIGS. 14A and 14B are block diagrams illustrating operation of awake-and-go engine with look-ahead in accordance with an illustrativeembodiment;

FIG. 15 is a flowchart illustrating a look-ahead polling operation of awake-and-go look-ahead engine in accordance with an illustrativeembodiment;

FIG. 16 is a block diagram illustrating operation of a wake-and-gomechanism with speculative execution in accordance with an illustrativeembodiment;

FIG. 17 is a flowchart illustrating operation of a look-aheadwake-and-go mechanism with speculative execution in accordance with anillustrative embodiment;

FIGS. 18A and 18B are flowcharts illustrating operation of a wake-and-gomechanism with speculative execution during execution of a thread inaccordance with an illustrative embodiment;

FIG. 19 is a block diagram illustrating data monitoring in a multipleprocessor system in accordance with an illustrative embodiment;

FIG. 20 is a block diagram illustrating operation of a wake-and-gomechanism in accordance with an illustrative embodiment;

FIGS. 21A and 21B are block diagrams illustrating parallel lock spinningusing a wake-and-go mechanism in accordance with an illustrativeembodiment;

FIGS. 22A and 22B are flowcharts illustrating parallel lock spinningusing a wake-and-go mechanism in accordance with the illustrativeembodiments;

FIG. 23 is a block diagram illustrating a wake-and-go engine with acentral repository wake-and-go array in a multiple processor system inaccordance with an illustrative embodiment;

FIG. 24 illustrates a central repository wake-and-go-array in accordancewith an illustrative embodiment;

FIG. 25 is a block diagram illustrating a programming idiom acceleratorin accordance with an illustrative embodiment;

FIG. 26 is a series of instructions that are a programming idiom withprogramming language exposure in accordance with an illustrativeembodiment;

FIG. 27 is a block diagram illustrating a compiler that exposesprogramming idioms in accordance with an illustrative embodiment; and

FIG. 28 is a flowchart illustrating operation of a compiler exposingprogramming idioms in accordance with an illustrative embodiment;

FIG. 29 is a block diagram illustrating a data processing system with aremote update programming idiom accelerator in accordance with anillustrative embodiment;

FIGS. 30A and 30B are flowcharts illustrating operation of a remoteupdate in accordance with the illustrative embodiments; and

FIGS. 31A and 31B are flowcharts illustrating operation of a remoteupdate with dynamic instruction size in accordance with the illustrativeembodiments.

DETAILED DESCRIPTION

With reference now to the figures and in particular with reference toFIG. 1, an exemplary diagram of data processing environments is providedin which illustrative embodiments of the present invention may beimplemented. It should be appreciated that FIG. 1 is only exemplary andis not intended to assert or imply any limitation with regard to theenvironments in which aspects or embodiments of the present inventionmay be implemented. Many modifications to the depicted environments maybe made without departing from the spirit and scope of the presentinvention.

FIG. 1 is a block diagram of an exemplary data processing system inwhich aspects of the illustrative embodiments may be implemented. Asshown, data processing system 100 includes processor cards 111 a-111 n.Each of processor cards 111 a-111 n includes a processor and a cachememory. For example, processor card 111 a contains processor 112 a andcache memory 113 a, and processor card 111 n contains processor 112 nand cache memory 113 n.

Processor cards 111 a-111 n connect to symmetric multiprocessing (SMP)bus 115. SMP bus 115 supports a system planar 120 that containsprocessor cards 111 a-111 n and memory cards 123. The system planar alsocontains data switch 121 and memory controller/cache 122. Memorycontroller/cache 122 supports memory cards 123 that includes localmemory 116 having multiple dual in-line memory modules (DIMMs).

Data switch 121 connects to bus bridge 117 and bus bridge 118 locatedwithin a native input/output (I/O) (NIO) planar 124. As shown, busbridge 118 connects to peripheral components interconnect (PCI) bridges125 and 126 via system bus 119. PCI bridge 125 connects to a variety ofI/O devices via PCI bus 128. As shown, hard disk 136 may be connected toPCI bus 128 via small computer system interface (SCSI) host adapter 130.A graphics adapter 131 may be directly or indirectly connected to PCIbus 128. PCI bridge 126 provides connections for external data streamsthrough network adapter 134 and adapter card slots 135 a-135 n via PCIbus 127.

An industry standard architecture (ISA) bus 129 connects to PCI bus 128via ISA bridge 132. ISA bridge 132 provides interconnection capabilitiesthrough NIO controller 133 having serial connections Serial 1 and Serial2. A floppy drive connection 137, keyboard connection 138, and mouseconnection 139 are provided by NIO controller 133 to allow dataprocessing system 100 to accept data input from a user via acorresponding input device. In addition, non-volatile random accessmemory (RAM) (NVRAM) 140 provides a non-volatile memory for preservingcertain types of data from system disruptions or system failures, suchas power supply problems. A system firmware 141 also connects to ISA bus129 for implementing the initial Basic Input/Output System (BIOS)functions. A service processor 144 connects to ISA bus 129 to providefunctionality for system diagnostics or system servicing.

The operating system (OS) resides on hard disk 136, which may alsoprovide storage for additional application software for execution bydata processing system. NVRAM 140 stores system variables and errorinformation for field replaceable unit (FRU) isolation. During systemstartup, the bootstrap program loads the operating system and initiatesexecution of the operating system. To load the operating system, thebootstrap program first locates an operating system kernel type fromhard disk 136, loads the OS into memory, and jumps to an initial addressprovided by the operating system kernel. Typically, the operating systemloads into random-access memory (RAM) within the data processing system.Once loaded and initialized, the operating system controls the executionof programs and may provide services such as resource allocation,scheduling, input/output control, and data management.

The present invention may be executed in a variety of data processingsystems utilizing a number of different hardware configurations andsoftware such as bootstrap programs and operating systems. The dataprocessing system 100 may be, for example, a stand-alone system or partof a network such as a local-area network (LAN) or a wide-area network(WAN).

FIG. 1 is an example of a symmetric multiprocessing (SMP) dataprocessing system in which processors communicate via a SMP bus 115.FIG. 1 is only exemplary and is not intended to assert or imply anylimitation with regard to the environments in which aspects orembodiments of the present invention may be implemented. The depictedenvironments may be implemented in other data processing environmentswithout departing from the spirit and scope of the present invention.

FIG. 2 is a block diagram of a wake-and-go mechanism in a dataprocessing system in accordance with an illustrative embodiment. Threads202, 204, 206 run on one or more processors (not shown). Threads 202,204, 206 make calls to operating system 210 and application programminginterface (API) 212 to communicate with each other, memory 232 via bus220, or other devices within the data processing system.

In accordance with the illustrative embodiment, a wake-and-go mechanismfor a microprocessor includes wake-and-go array 222 attached to the SMPfabric. The SMP fabric is a communication medium through whichprocessors communicate. The SMP fabric may comprise a single SMP bus ora system of busses, for example. In the depicted example, the SMP fabriccomprises bus 220. A thread, such as thread 202, for example, mayinclude instructions that indicate that the thread is waiting for anevent. The event may be an asynchronous event, which is an event thathappens independently in time with respect to execution of the thread inthe data processing system. For example, an asynchronous event may be atemperature value reaching a particular threshold, a stock price fallingbelow a given threshold, or the like. Alternatively, the event may berelated in some way to execution of the thread. For example, the eventmay be obtaining a lock for exclusive access to a database record or thelike.

Typically, the instructions may comprise a series of get-and-comparesequences; however, in accordance with the illustrative embodiment, theinstructions include instructions, calls to operating system 210 or API212, or calls to a background sleeper thread, such as thread 204, forexample, to update wake-and-go array 222. These instructions store atarget address in wake-and-go array 222, where the event the thread iswaiting for is associated with the target address. After updatingwake-and-go array 222 with the target address, thread 202 may go tosleep.

When thread 202 goes to sleep, operating system 210 or other software orhardware saves the state of thread 202 in thread state storage 234,which may be allocated from memory 232 or may be a hardware privatearray within the processor (not shown) or pervasive logic (not shown).When a thread is put to sleep, i.e., removed from the run queue of aprocessor, the operating system must store sufficient information on itsoperating state such that when the thread is again scheduled to run onthe processor, the thread can resume operation from an identicalposition. This state information is sometime referred to as the thread's“context.” The state information may include, for example, addressspace, stack space, virtual address space, program counter, instructionregister, program status word, and the like.

If a transaction appears on bus 220 that modifies a value at an addressin wake-and-go array 222, then operating system 210 may wake thread 202.Operating system 210 wakes thread 202 by recovering the state of thread202 from thread state storage 234. Thread 202 may then determine whetherthe transaction corresponds to the event for which the thread waswaiting by performing a get-and-compare operation, for instance. If thetransaction is the event for which the thread was waiting, then thread202 will perform work. However, if the transaction is not the event,then thread 202 will go back to sleep. Thus, thread 202 only performs aget-and-compare operation if there is a transaction that modifies thetarget address.

Alternatively, operating system 210 or a background sleeper thread, suchas thread 204, may determine whether the transaction is the event forwhich the thread was waiting. Before being put to sleep, thread 202 mayupdate a data structure in the operating system or background sleeperthread with a value for which it is waiting.

In one exemplary embodiment, wake-and-go array 222 may be a contentaddressable memory (CAM). A CAM is a special type of computer memoryoften used in very high speed searching applications. A CAM is alsoknown as associative memory, associative storage, or associative array,although the last term is more often used for a programming datastructure. Unlike a random access memory (RAM) in which the usersupplies a memory address and the RAM returns the data value stored atthat address, a CAM is designed such that the user supplies a data valueand the CAM searches its entire memory to see if that data value isstored within the CAM. If the data value is found, the CAM returns alist of one or more storage addresses where the data value was found. Insome architectures, a CAM may return the data value or other associatedpieces of data. Thus, a CAM may be considered the hardware embodiment ofwhat in software terms would be called an associative array.

Thus, in the exemplary embodiment, wake-and-go array 222 may comprise aCAM and associated logic that will be triggered if a transaction appearson bus 220 that modifies an address stored in the CAM. A transactionthat modifies a value at a target address may be referred to as a“kill”; thus, wake-and-go array 222 may be said to be “snooping kills.”In this exemplary embodiment, the data values stored in the CAM are thetarget addresses at which threads are waiting for something to bewritten. The address at which a data value, a given target address, isstored is referred to herein as the storage address. Each storageaddress may refer to a thread that is asleep and waiting for an event.Wake-and-go array 222 may store multiple instances of the same targetaddress, each instance being associated with a different thread waitingfor an event at that target address. Thus, when wake-and-go array 222snoops a kill at a given target address, wake-and-go array 222 mayreturn one or more storage addresses that are associated with one ormore sleeping threads.

In one exemplary embodiment, software may save the state of thread 202,for example. The state of a thread may be about 1000 bytes, for example.Thread 202 is then put to sleep. When wake-and-go array 222 snoops akill at a given target address, logic associated with wake-and-go array222 may generate an exception. The processor that was running thread 202sees the exception and performs a trap. A trap is a type of synchronousinterrupt typically caused by an exception condition, in this case akill at a target address in wake-and-go array 222. The trap may resultin a switch to kernel mode, wherein the operating system 210 performssome action before returning control to the originating process. In thiscase, the trap results in other software, such as operating system 210,for example, to reload thread 202 from thread state storage 234 and tocontinue processing of the active threads on the processor.

FIG. 3 is a block diagram of a wake-and-go mechanism with a hardwareprivate array in accordance with an illustrative embodiment. Threads302, 304, 306 run on processor 300. Threads 302, 304, 306 make calls tooperating system 310 and application programming interface (API) 312 tocommunicate with each other, memory 332 via bus 320, or other deviceswithin the data processing system. While the data processing system inFIG. 3 shows one processor, more processors may be present dependingupon the implementation where each processor has a separate wake-and-goarray or one wake-and-go array stores target addresses for threads formultiple processors.

In an illustrative embodiment, when a thread, such as thread 302, firststarts executing, a wake-and-go mechanism automatically allocates spacefor thread state in hardware private array 308 and space for a targetaddress and other information, if any, in wake-and-go array 322.Allocating space may comprise reserving an address range in a memory,such as a static random access memory, that is hidden in hardware, suchas processor 300, for example. Alternatively, if hardware private array308 comprises a reserved portion of system memory, such as memory 332,then the wake-and-go mechanism may request a sufficient portion ofmemory, such as 1000 bytes, for example, to store thread state for thatthread.

Thus hardware private array 308 may be a memory the size of whichmatches the size of thread state information for all running threads.When a thread ends execution and is no longer in the run queue ofprocessor 300, the wake-and-go mechanism de-allocates the space for thethread state information for that thread.

In accordance with the illustrative embodiment, a wake-and-go mechanismfor a microprocessor includes wake-and-go array 322 attached to the SMPfabric. The SMP fabric is a communication medium through whichprocessors communicate. The SMP fabric may comprise a single SMP bus ora system of busses, for example. In the depicted example, the SMP fabriccomprises bus 320. A thread, such as thread 302, for example, mayinclude instructions that indicate that the thread is waiting for anevent. The event may be an asynchronous event, which is an event thathappens independently in time with respect to execution of the thread inthe data processing system. For example, an asynchronous event may be atemperature value reaching a particular threshold, a stock price fallingbelow a given threshold, or the like. Alternatively, the event may berelated in some way to execution of the thread. For example, the eventmay be obtaining a lock for exclusive access to a database record or thelike.

Typically, the instructions may comprise a series of get-and-comparesequences; however, in accordance with the illustrative embodiment, theinstructions include instructions, calls to operating system 310 or API312, or calls to a background sleeper thread, such as thread 304, forexample, to update wake-and-go array 322. These instructions store atarget address in wake-and-go array 322, where the event the thread iswaiting for is associated with the target address. After updatingwake-and-go array 322 with the target address, thread 302 may go tosleep.

When thread 302 goes to sleep, operating system 310 or other software orhardware within processor 300 saves the state of thread 302 in hardwareprivate array 308 within processor 300. In an alternative embodiment,hardware private array may be embodied within pervasive logic associatedwith bus 320 or wake-and-go array 322. When a thread is put to sleep,i.e., removed from the run queue of processor 300, operating system 310must store sufficient information on its operating state such that whenthe thread is again scheduled to run on processor 300, the thread canresume operation from an identical position. This state information issometime referred to as the thread's “context.” The state informationmay include, for example, address space, stack space, virtual addressspace, program counter, instruction register, program status word, andthe like, which may comprise about 1000 bytes, for example.

If a transaction appears on bus 320 that modifies a value at an addressin wake-and-go array 322, then operating system 310 may wake thread 302.Operating system 310 wakes thread 302 by recovering the state of thread302 from hardware private array 308. Thread 302 may then determinewhether the transaction corresponds to the event for which the threadwas waiting by performing a get-and-compare operation, for instance. Ifthe transaction is the event for which the thread was waiting, thenthread 302 will perform work. However, if the transaction is not theevent, then thread 302 will go back to sleep. Thus, thread 302 onlyperforms a get-and-compare operation if there is a transaction thatmodifies the target address.

Hardware private array 308 is a thread state storage that is embeddedwithin processor 300 or within logic associated with bus 320 orwake-and-go array 322. Hardware private array 308 may be a memorystructure, such as a static random access memory (SRAM), which isdedicated to storing thread state for sleeping threads that have atarget address in wake-and-go array 322. In an alternative embodiment,hardware private array 308 may be a hidden area of memory 332. Hardwareprivate array 308 is private because it cannot be addressed by theoperating system or work threads.

Hardware private array 308 and/or wake-and-go array 322 may have alimited storage area. Therefore, each thread may have an associatedpriority. The wake-and-go mechanism described herein may store thepriority of sleeping threads with the thread state in hardware privatearray 308. Alternatively, the wake-and-go mechanism may store thepriority with the target address in wake-and-go array 322. When athread, such as thread 302, for example, goes to sleep, the wake-and-gomechanism may determine whether there is sufficient room to store thethread state of thread 302 in hardware private array 308. If there issufficient space, then the wake-and-go mechanism simply stores thethread state in hardware private array 308.

If there is insufficient space in hardware private array 308, then ifthe hardware private array is a portion of system memory 332, then thewake-and-go mechanism may ask for more of system memory 332 to beallocated to the hardware private array 308.

If there is insufficient space in hardware private array 308, then thewake-and-go mechanism may compare the priority of thread 302 to thepriorities of the threads already stored in hardware private array 308and wake-and-go array 322. If thread 302 has a lower priority than allof the threads already stored in hardware private array 208 andwake-and-go array 322, then thread 302 may default to a flee model, suchas polling or interrupt as in the prior art. If thread 302 has a higherpriority than at least one thread already stored in hardware privatearray 308 and wake-and-go array 322, then the wake-and-go mechanism may“punt” a lowest priority thread, meaning the thread is removed fromhardware private array 308 and wake-and-go array 322 and converted to aflee model.

In an alternative embodiment, priority may be determined by otherfactors. For example, priority may be time driven. That is, thewake-and-go mechanism may simply punt the stalest thread in hardwareprivate array 308 and wake-and-go array 322.

Alternatively, operating system 310 or a background sleeper thread, suchas thread 304, may determine whether the transaction is the event forwhich the thread was waiting. Before being put to sleep, thread 302 mayupdate a data structure in the operating system or background sleeperthread with a value for which it is waiting.

In one exemplary embodiment, wake-and-go array 322 may be a contentaddressable memory (CAM). A CAM is a special type of computer memoryoften used in very high speed searching applications. A CAM is alsoknown as associative memory, associative storage, or associative array,although the last term is more often used for a programming datastructure. Unlike a random access memory (RAM) in which the usersupplies a memory address and the RAM returns the data value stored atthat address, a CAM is designed such that the user supplies a data valueand the CAM searches its entire memory to see if that data value isstored within the CAM. If the data value is found, the CAM returns alist of one or more storage addresses where the data value was found. Insome architectures, a CAM may return the data value or other associatedpieces of data. Thus, a CAM may be considered the hardware embodiment ofwhat in software terms would be called an associative array.

Thus, in the exemplary embodiment, wake-and-go array 322 may comprise aCAM and associated logic that will be triggered if a transaction appearson bus 320 that modifies an address stored in the CAM. A transactionthat modifies a value at a target address may be referred to as a“kill”; thus, wake-and-go array 322 may be said to be “snooping kills.”In this exemplary embodiment, the data values stored in the CAM are thetarget addresses at which threads are waiting for something to bewritten. The address at which a data value, a given target address, isstored is referred to herein as the storage address. Each storageaddress may refer to a thread that is asleep and waiting for an event.Wake-and-go array 322 may store multiple instances of the same targetaddress, each instance being associated with a different thread waitingfor an event at that target address. Thus, when wake-and-go array 322snoops a kill at a given target address, wake-and-go array 322 mayreturn one or more storage addresses that are associated with one ormore sleeping threads.

FIGS. 4A and 4B are block diagrams illustrating operation of awake-and-go mechanism with specialized processor instructions inaccordance with an illustrative embodiment. With particular reference toFIG. 4A, thread 410 runs in a processor (not shown) and performs somework. Thread 410 executes a specialized processor instruction to updatewake-and-go array 422, storing a target address A₂ in array 422. Then,thread 410 goes to sleep with thread state being stored in thread statestorage 412.

When a transaction appears on SMP fabric 420 with an address thatmatches the target address A₂, array 422 returns the storage addressthat is associated with thread 410. The operating system (not shown) orsome other hardware or software then wakes thread 410 by retrieving thethread state information from thread state storage 412 and placing thethread in the run queue for the processor. Thread 410 may then perform acompare-and-branch operation to determine whether the value written tothe target address represents the event for which thread 410 is waiting.In the depicted example, the value written to the target address doesnot represent the event for which thread 410 is waiting; therefore,thread 410 goes back to sleep.

In one exemplary embodiment, software may save the state of thread 410,for example. Thread 410 is then put to sleep. When wake-and-go array 422snoops a kill at target address A₂, logic associated with wake-and-goarray 422 may generate an exception. The processor sees the exceptionand performs a trap, which results in a switch to kernel mode, whereinthe operating system may perform some action before returning control tothe originating process. In this case, the trap results in othersoftware to reload thread 410 from thread state storage 412 and tocontinue processing of the active threads on the processor.

In one exemplary embodiment, thread state storage 412 is a hardwareprivate array. Thread state storage 412 is a memory that is embeddedwithin the processor or within logic associated with bus 420 orwake-and-go array 422. Thread state storage 412 may comprise memorycells that are dedicated to storing thread state for sleeping threadsthat have a target address in wake-and-go array 422. In an alternativeembodiment, thread state storage 412 may be a hidden area of memory 332,for example. Thread state storage 412 may private in that it cannot beaddressed by the operating system or work threads.

Turning to FIG. 4B, thread 410 runs in a processor (not shown) andperforms some work. Thread 410 executes a specialized processorinstruction to update wake-and-go array 422, storing a target address A₂in array 422. Then, thread 410 goes to sleep with thread state beingstored in thread state storage 412.

When a transaction appears on SMP fabric 420 with an address thatmatches the target address A₂, array 422 returns the storage addressthat is associated with thread 410. The operating system (not shown) orsome other hardware or software then wakes thread 410 by retrieving thethread state information from thread state storage 412 and placing thethread in the run queue for the processor. Thread 410 may then perform acompare-and-branch operation to determine whether the value written tothe target address represents the event for which thread 410 is waiting.In the depicted example, the value written to the target address doesrepresent the event for which thread 410 is waiting; therefore, thread410 updates the array to remove the target address from array 422, andperforms more work.

FIGS. 5A and 5B are block diagrams illustrating operation of awake-and-go mechanism with a specialized operating system call inaccordance with an illustrative embodiment. With particular reference toFIG. 5A, thread 510 runs in a processor (not shown) and performs somework. Thread 510 makes a call to operating system 530 to updatewake-and-go array 522. The call to operating system 530 may be anoperating system call or a call to an application programming interface(not shown) provided by operating system 530. Operating system 530 thenstores a target address A₂ in array 522. Then, thread 510 goes to sleepwith thread state being stored in thread state storage 512.

When a transaction appears on SMP fabric 520 with an address thatmatches the target address A₂, array 522 returns the storage addressthat is associated with thread 510. Operating system 530 or some otherhardware or software then wakes thread 510 by retrieving the threadstate information from thread state storage 512 and placing the threadin the run queue for the processor. Thread 510 may then perform acompare-and-branch operation to determine whether the value written tothe target address represents the event for which thread 510 is waiting.In the depicted example, the value written to the target address doesnot represent the event for which thread 510 is waiting; therefore,thread 510 goes back to sleep.

In one exemplary embodiment, software may save the state of thread 510,for example. Thread 510 is then put to sleep. When wake-and-go array 522snoops a kill at target address A₂, logic associated with wake-and-goarray 522 may generate an exception. The processor sees the exceptionand performs a trap, which results in a switch to kernel mode, whereinoperating system 530 may perform some action before returning control tothe originating process. In this case, the trap results in the operatingsystem 530 to reload thread 510 from thread state storage 512 and tocontinue processing of the active threads on the processor.

In one exemplary embodiment, thread state storage 512 is a hardwareprivate array. Thread state storage 512 is a memory that is embeddedwithin the processor or within logic associated with bus 520 orwake-and-go array 522. Thread state storage 512 may comprise memorycells that are dedicated to storing thread state for sleeping threadsthat have a target address in wake-and-go array 522. In an alternativeembodiment, thread state storage 512 may be a hidden area of memory 332,for example. Thread state storage 512 may private in that it cannot beaddressed by the operating system or work threads.

Turning to FIG. 5B, thread 510 runs in a processor (not shown) andperforms some work. Thread 510 makes a call to operating system 530 toupdate wake-and-go array 522. The call to operating system 530 may be anoperating system call or a call to an application programming interface(not shown) provided by operating system 530. Operating system 530 thenstores a target address A₂ in array 522. Then, thread 510 goes to sleepwith thread state being stored in thread state storage 512.

When a transaction appears on SMP fabric 520 with an address thatmatches the target address A₂, array 522 returns the storage addressthat is associated with thread 510. Operating system 530 or some otherhardware or software then wakes thread 510 by retrieving the threadstate information from thread state storage 512 and placing the threadin the run queue for the processor. Thread 510 may then perform acompare-and-branch operation to determine whether the value written tothe target address represents the event for which thread 510 is waiting.In the depicted example, the value written to the target address doesrepresent the event for which thread 510 is waiting; therefore, thread510 updates the array to remove the target address from array 522, andperforms more work.

FIG. 6 is a block diagram illustrating operation of a wake-and-gomechanism with a background sleeper thread in accordance with anillustrative embodiment. Thread 610 runs in a processor (not shown) andperforms some work. Thread 610 makes a call to background sleeper thread640 to update wake-and-go array 622. The call to background sleeperthread 640 may be a remote procedure call, for example, or a call to anapplication programming interface (not shown) provided by backgroundsleeper thread 640. Background sleeper thread 640 then stores a targetaddress A₂ in array 622. Thread 610 may also store other information inassociation with background sleeper thread 640, such as a value forwhich thread 610 is waiting to be written to target address A₂. Then,thread 610 goes to sleep with thread state being stored in thread statestorage 612.

When a transaction appears on SMP fabric 620 with an address thatmatches the target address A₂, array 622 returns the storage addressthat is associated with thread 610. Operating system 630 or some otherhardware or software then wakes thread 610 by retrieving the threadstate information from thread state storage 612 and placing the threadin the run queue for the processor. Background sleeper thread 640 maythen perform a compare-and-branch operation to determine whether thevalue written to the target address represents the event for whichthread 610 is waiting. If the value written to the target address doesrepresent the event for which thread 610 is waiting, then backgroundsleeper thread 640 does nothing. However, if the value written to thetarget address does represent the event for which thread 610 is waiting,then background sleeper thread 640 wakes thread 640. Thereafter, thread610 updates the array 622 to remove the target address from array 622and performs more work.

In one exemplary embodiment, software may save the state of thread 610,for example. Thread 610 is then put to sleep. When wake-and-go array 622snoops a kill at target address A₂, logic associated with wake-and-goarray 622 may generate an exception. The processor sees the exceptionand performs a trap, which results in a switch to kernel mode, whereinthe operating system may perform some action before returning control tothe originating process. In this case, the trap results in othersoftware, such as background sleeper thread 640 to reload thread 610from thread state storage 612 and to continue processing of the activethreads on the processor.

In one exemplary embodiment, thread state storage 612 is a hardwareprivate array. Thread state storage 612 is a memory that is embeddedwithin the processor or within logic associated with bus 620 orwake-and-go array 622. Thread state storage 612 may comprise memorycells that are dedicated to storing thread state for sleeping threadsthat have a target address in wake-and-go array 622. In an alternativeembodiment, thread state storage 612 may be a hidden area of memory 332,for example. Thread state storage 612 may private in that it cannot beaddressed by the operating system or work threads.

As will be appreciated by one skilled in the art, the present inventionmay be embodied as a system, method, or computer program product.Accordingly, the present invention may take the form of an entirelyhardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit,” “module” or “system.” Furthermore,the present invention may take the form of a computer program productembodied in any tangible medium of expression having computer usableprogram code embodied in the medium.

Any combination of one or more computer usable or computer readablemedium(s) may be utilized. The computer-usable or computer-readablemedium may be, for example, but not limited to, an electronic, magnetic,optical, electromagnetic, infrared, or semiconductor system, apparatus,device, or propagation medium. More specific examples (a non-exhaustivelist) of the computer-readable medium would include the following: anelectrical connection having one or more wires, a portable computerdiskette, a hard disk, a random access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), an optical fiber, a portable compact disc read-only memory(CDROM), an optical storage device, a transmission media such as thosesupporting the Internet or an intranet, or a magnetic storage device.Note that the computer-usable or computer-readable medium could even bepaper or another suitable medium upon which the program is printed, asthe program can be electronically captured, via, for instance, opticalscanning of the paper or other medium, then compiled, interpreted, orotherwise processed in a suitable manner, if necessary, and then storedin a computer memory. In the context of this document, a computer-usableor computer-readable medium may be any medium that can contain, store,communicate, propagate, or transport the program for use by or inconnection with the instruction execution system, apparatus, or device.The computer-usable medium may include a propagated data signal with thecomputer-usable program code embodied therewith, either in baseband oras part of a carrier wave. The computer usable program code may betransmitted using any appropriate medium, including but not limited towireless, wireline, optical fiber cable, radio frequency (RF), etc.

Computer program code for carrying out operations of the presentinvention may be written in any combination of one or more programminglanguages, including an object oriented programming language such asJava™, Smalltalk™, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider). In addition, the program code may be embodied on a computerreadable storage medium on the server or the remote computer anddownloaded over a network to a computer readable storage medium of theremote computer or the users' computer for storage and/or execution.Moreover, any of the computing systems or data processing systems maystore the program code in a computer readable storage medium afterhaving downloaded the program code over a network from a remotecomputing system or data processing system.

The illustrative embodiments are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to the illustrativeembodiments of the invention. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in acomputer-readable medium that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablemedium produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide processes for implementing the functions/actsspecified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof code, which comprises one or more executable instructions forimplementing the specified logical function(s). It should also be notedthat, in some alternative implementations, the functions noted in theblock may occur out of the order noted in the figures. For example, twoblocks shown in succession may, in fact, be executed substantiallyconcurrently, or the blocks may sometimes be executed in the reverseorder, depending upon the functionality involved. It will also be notedthat each block of the block diagrams and/or flowchart illustration, andcombinations of blocks in the block diagrams and/or flowchartillustration, can be implemented by special purpose hardware-basedsystems that perform the specified functions or acts, or combinations ofspecial purpose hardware and computer instructions.

FIGS. 7A and 7B are flowcharts illustrating operation of a wake-and-gomechanism in accordance with the illustrative embodiments. Withreference now to FIG. 7A, operation begins when a thread firstinitializes or when a thread wakes after sleeping. The operating systemstarts a thread (block 702) by initializing the thread and placing thethread in the run queue for a processor. The thread then performs work(block 704). The operating system determines whether the thread hascompleted (block 706). If the thread completes, then operation ends.

If the end of the thread is not reached in block 706, the processordetermines whether the next instruction updates the wake-and-go array(block 708). An instruction to update the wake-and-go array may be aspecialized processor instruction, an operating system call, a call to abackground sleeper thread, or a call to an application programminginterface. If the next instruction does not update the wake-and-goarray, operation returns to block 704 to perform more work.

If the next instruction does update the wake-and-go array in block 708,the processor updates the array with a target address associated with anevent for which the thread is waiting (block 710). The update to thewake-and-go array may be made by the thread through a specializedprocessor instruction, the operating system, or a background sleeperthread. Next, the operating system then determines whether to put thethread to sleep (block 712). The operating system may keep the threadactive in the processor if the processor is underutilized, for instance;however, the operating system may put the thread to sleep if there areother threads waiting to be run on the processor. If the operatingsystem determines that the thread is to remain active, operation returnsto block 704 to perform more work, in which case the thread may simplywait for the event.

In one exemplary embodiment, if the operating system determines that thethread is to be put to sleep in block 712, then the operating system orsome other software or hardware saves the state of the thread (block714) and puts the thread to sleep (block 716). Thereafter, operationproceeds to FIG. 7B where the wake-and-go mechanism monitors for anevent. In one exemplary embodiment, software may save the state of thethread in thread state storage. The thread is then put to sleep.

In an alternative embodiment, if the operating system determines thatthe thread is to be put to sleep in block 712, then the operating systemor some other software or hardware saves the state of the thread (block714) in the hardware private array and puts the thread to sleep (block716). Thereafter, operation proceeds to FIG. 7B where the wake-and-gomechanism monitors for an event.

With reference now to FIG. 7B, the wake-and-go mechanism, which mayinclude a wake-and-go array, such as a content addressable memory, andassociated logic, snoops for a kill from the symmetric multiprocessing(SMP) fabric (block 718). A kill occurs when a transaction appears onthe SMP fabric that modifies the target address associated with theevent for which a thread is waiting. The wake-and-go mechanism thenperforms a compare (block 720) and determines whether the value beingwritten to the target address represents the event for which the threadis waiting (block 722). If the kill corresponds to the event for whichthe thread is waiting, then the operating system updates the array(block 724) to remove the target address from the wake-and-go array.Thereafter, operation returns to block 702 in FIG. 7A where theoperating system restarts the thread.

In one exemplary embodiment, when the wake-and-go mechanism snoops akill at a target address, the wake-and-go mechanism may generate anexception. The processor sees the exception and performs a trap, whichresults in a switch to kernel mode, wherein the operating system mayperform some action before returning control to the originating process.In this case, the trap results in other software to reload the threadfrom the thread state storage and to continue processing of the activethreads on the processor in block 702.

In one exemplary embodiment, when the wake-and-go mechanism snoops akill at a target address, software or hardware reloads the thread fromthe hardware private array and the processor continues processing theactive threads on the processor in block 702.

If the kill does not correspond to the event for which the thread iswaiting in block 722, then operation returns to block 718 to snoop akill from the SMP fabric. In FIG. 7B, the wake-and-go mechanism may be acombination of logic associated with the wake-and-go array, such as aCAM, and software within the operating system, software within abackground sleeper thread, or other hardware.

In an alternative embodiment, the wake-and-go mechanism may be acombination of logic associated with the wake-and-go array and softwarewithin the thread itself. In such an embodiment, the thread will wakeevery time there is a kill to the target address. The thread itself maythen perform a compare operation to determine whether to perform morework or to go back to sleep. If the thread decides to go back to sleep,it may again save the state of the thread. The over head for waking thethread every time there is a kill to the target address will likely bemuch less than polling or event handlers.

Prioritization of Threads

FIGS. 8A and 8B are flowcharts illustrating operation of a wake-and-gomechanism with prioritization of threads in accordance with theillustrative embodiments. Operation begins when a thread firstinitializes or when a thread wakes after sleeping. The operating systemstarts a thread (block 802) by initializing the thread and placing thethread in the run queue for a processor. The thread then performs work(block 804). The operating system determines whether the thread hascompleted (block 806). If the thread completes, then operation ends.

If the end of the thread is not reached in block 806, the processordetermines whether the next instruction updates the wake-and-go array(block 808). An instruction to update the wake-and-go array may be aspecialized processor instruction, an operating system call, a call to abackground sleeper thread, or a call to an application programminginterface. If the next instruction does not update the wake-and-goarray, operation returns to block 804 to perform more work.

If the next instruction does update the wake-and-go array in block 808,the wake-and-go mechanism determines whether there is sufficient spacefor the thread state in the hardware private array (block 810). If thereis sufficient space available, the wake-and-go mechanism allocates spacefor the thread state in the hardware private array (block 812). Thisallocation may simply comprise reserving the requisite space for thethread space, which may be about 1000 bytes, for example. If thehardware private array is reserved portion of system memory, thenallocating space may comprise requesting more system memory to bereserved for the hardware private array. Then, the wake-and-go mechanismsaves the state of the thread in the hardware private array (block 814),updates the wake-and-go array with the target address and otherinformation, if any (block 816), and puts the thread to sleep (block818). Thereafter, operation proceeds to FIG. 8B where the wake-and-gomechanism monitors for an event.

If there is insufficient space for the thread state available in thehardware private array in block 810, then the wake-and-go mechanismdetermines whether there is at least one lower priority thread in thehardware private array or wake-and-go array (block 820). As describedabove, each thread may have an associated priority parameter that isstored in the hardware private array or wake-and-go array.Alternatively, priority may be determined by other factors, such asstaleness. If there is at least one lower priority thread in thehardware private array, the wake-and-go mechanism removes the lowerpriority thread from the hardware private array and wake-and-go array(block 822) and converts the lower priority thread to a flee model(block 824). Thereafter, operation proceeds to block 814 to save thestate of the new thread, update the wake-and-go array, and put thethread to sleep.

If there is not a lower priority thread in the hardware private array inblock 820, the wake-and-go mechanism converts the new thread to a fleemodel (block 826). Thereafter, operation proceeds to block 818 to putthe thread to sleep.

With reference now to FIG. 8B, the wake-and-go mechanism, which mayinclude a wake-and-go array, such as a content addressable memory, andassociated logic, snoops for a kill from the symmetric multiprocessing(SMP) fabric (block 826). A kill occurs when a transaction appears onthe SMP fabric that modifies the target address associated with theevent for which a thread is waiting. The wake-and-go mechanism thenperforms a compare (block 828) and determines whether the value beingwritten to the target address represents the event for which the threadis waiting (block 830). If the kill corresponds to the event for whichthe thread is waiting, then the operating system updates the wake-and-goarray (block 832) to remove the target address from the wake-and-goarray. Then, the wake-and-go mechanism reloads the thread from thehardware private array (block 834). Thereafter, operation returns toblock 802 in FIG. 8A where the operating system restarts the thread.

In one exemplary embodiment, when the wake-and-go mechanism snoops akill at a target address, software or hardware reloads the thread fromthe hardware private array and the processor continues processing theactive threads on the processor in block 802.

If the kill does not correspond to the event for which the thread iswaiting in block 830, then operation returns to block 826 to snoop akill from the SMP fabric. In FIG. 8B, the wake-and-go mechanism may be acombination of logic associated with the wake-and-go array, such as aCAM, and software within the operating system, software within abackground sleeper thread, or other hardware.

Dynamic Allocation in Hardware Private Array

FIGS. 9A and 9B are flowcharts illustrating operation of a wake-and-gomechanism with dynamic allocation in a hardware private array inaccordance with the illustrative embodiments. Operation begins when athread first initializes or when a thread wakes after sleeping. Thewake-and-go mechanism allocates space for thread state information inthe hardware private array (block 902). The operating system starts athread (block 904) by initializing the thread and placing the thread inthe run queue for a processor. The wake-and-go mechanism may alsoallocate space in the wake-and-go array. The thread then performs work(block 906). The operating system determines whether the thread hascompleted (block 908). If the thread completes, then the wake-and-gomechanism de-allocates the space corresponding to the thread stateinformation for the thread (block 910), and operation ends.

If the end of the thread is not reached in block 908, the processordetermines whether the next instruction updates the wake-and-go array(block 912). An instruction to update the wake-and-go array may be aspecialized processor instruction, an operating system call, a call to abackground sleeper thread, or a call to an application programminginterface. If the next instruction does not update the wake-and-goarray, operation returns to block 906 to perform more work.

If the next instruction does update the wake-and-go array in block 912,the wake-and-go mechanism updates the wake-and-go array with a targetaddress associated with an event for which the thread is waiting (block914). The update to the wake-and-go array may be made by the threadthrough a specialized processor instruction, the operating system, or abackground sleeper thread. Next, the operating system then determineswhether to put the thread to sleep (block 916). The operating system maykeep the thread active in the processor if the processor isunderutilized, for instance; however, the operating system may put thethread to sleep if there are other threads waiting to be run on theprocessor. If the operating system determines that the thread is toremain active, operation returns to block 906 to perform more work, inwhich case the thread may simply wait for the event.

If the operating system determines that the thread is to be put to sleepin block 916, then the operating system or some other software orhardware saves the state of the thread (block 918) in the hardwareprivate array and puts the thread to sleep (block 920). Thereafter,operation proceeds to FIG. 9B where the wake-and-go mechanism monitorsfor an event.

With reference now to FIG. 9B, the wake-and-go mechanism, which mayinclude a wake-and-go array, such as a content addressable memory, andassociated logic, snoops for a kill from the symmetric multiprocessing(SMP) fabric (block 922). A kill occurs when a transaction appears onthe SMP fabric that modifies the target address associated with theevent for which a thread is waiting. The wake-and-go mechanism thenperforms a compare (block 924) and determines whether the value beingwritten to the target address represents the event for which the threadis waiting (block 926). If the kill corresponds to the event for whichthe thread is waiting, then the operating system updates the wake-and-goarray (block 928) to remove the target address from the wake-and-goarray. The wake-and-go mechanism then reloads the thread state from thehardware private array (block 930). Thereafter, operation returns toblock 904 in FIG. 9A where the operating system restarts the thread.

If the kill does not correspond to the event for which the thread iswaiting in block 922, then operation returns to block 922 to snoop akill from the SMP fabric. In FIG. 9B, the wake-and-go mechanism may be acombination of logic associated with the wake-and-go array, such as aCAM, and software within the operating system, software within abackground sleeper thread, or other hardware.

Hardware Wake-and-Go Mechanism

FIG. 10 is a block diagram of a hardware wake-and-go mechanism in a dataprocessing system in accordance with an illustrative embodiment. Threads1002, 1004, 1006 run on processor 1000. Threads 1002, 1004, 1006 makecalls to operating system 1010 to communicate with each other, memory1032 via bus 1020, or other devices within the data processing system.While the data processing system in FIG. 10 shows one processor, moreprocessors may be present depending upon the implementation where eachprocessor has a separate wake-and-go array or one wake-and-go arraystores target addresses for threads for multiple processors.

Wake-and-go mechanism 1008 is a hardware implementation within processor1000. In an alternative embodiment, hardware wake-and-go mechanism 1008may be logic associated with wake-and-go array 1022 attached to bus 1020or a separate, dedicated wake-and-go engine as described in furtherdetail below.

In accordance with the illustrative embodiment, hardware wake-and-gomechanism 1008 is provided within processor 1000 and wake-and-go array1022 is attached to the SMP fabric. The SMP fabric is a communicationmedium through which processors communicate. The SMP fabric may comprisea single SMP bus or a system of busses, for example. In the depictedexample, the SMP fabric comprises bus 1020. A thread, such as thread1002, for example, may include instructions that indicate that thethread is waiting for an event. The event may be an asynchronous event,which is an event that happens independently in time with respect toexecution of the thread in the data processing system. For example, anasynchronous event may be a temperature value reaching a particularthreshold, a stock price falling below a given threshold, or the like.Alternatively, the event may be related in some way to execution of thethread. For example, the event may be obtaining a lock for exclusiveaccess to a database record or the like.

Processor 1000 may pre-fetch instructions from storage (not shown) tomemory 1032. These instructions may comprise a get-and-compare sequence,for example. Wake-and-go mechanism 1008 within processor 1000 mayexamine the instruction stream as it is being pre-fetched and recognizethe get-and-compare sequence as a programming idiom that indicates thatthread 1002 is waiting for data at a particular target address. Aprogramming idiom is a sequence of programming instructions that occursoften and is recognizable as a sequence of instructions. In thisexample, an instruction sequence that includes load (LD), compare (CMP),and branch (BC) commands represents a programming idiom that indicatesthat the thread is waiting for data to be written to a particular targetaddress. In this case, wake-and-go mechanism 1008 recognizes such aprogramming idiom and may store the target address in wake-and-go array1022, where the event the thread is waiting for is associated with thetarget address. After updating wake-and-go array 1022 with the targetaddress, wake-and-go mechanism 1008 may put thread 1002 to sleep.

Wake-and-go mechanism 1008 also may save the state of thread 1002 inthread state storage 1034, which may be allocated from memory 1032 ormay be a hardware private array within the processor (not shown) orpervasive logic (not shown). When a thread is put to sleep, i.e.,removed from the run queue of a processor, the operating system muststore sufficient information on its operating state such that when thethread is again scheduled to run on the processor, the thread can resumeoperation from an identical position. This state information is sometimereferred to as the thread's “context.” The state information mayinclude, for example, address space, stack space, virtual address space,program counter, instruction register, program status word, and thelike.

If a transaction appears on bus 1020 that modifies a value at an addressin wake-and-go array 1022, then wake-and-go mechanism 1008 may wakethread 1002. Wake-and-go mechanism 1008 may wake thread 1002 byrecovering the state of thread 1002 from thread state storage 1034.Thread 1002 may then determine whether the transaction corresponds tothe event for which the thread was waiting by performing aget-and-compare operation, for instance. If the transaction is the eventfor which the thread was waiting, then thread 1002 will perform work.However, if the transaction is not the event, then thread 1002 will goback to sleep. Thus, thread 1002 only performs a get-and-compareoperation if there is a transaction that modifies the target address.

Alternatively, operating system 1010 or a background sleeper thread,such as thread 1004, may determine whether the transaction is the eventfor which the thread was waiting. Before being put to sleep, thread 1002may update a data structure in the operating system or backgroundsleeper thread with a value for which it is waiting.

In one exemplary embodiment, wake-and-go array 1022 may be a contentaddressable memory (CAM). A CAM is a special type of computer memoryoften used in very high speed searching applications. A CAM is alsoknown as associative memory, associative storage, or associative array,although the last term is more often used for a programming datastructure. Unlike a random access memory (RAM) in which the usersupplies a memory address and the RAM returns the data value stored atthat address, a CAM is designed such that the user supplies a data valueand the CAM searches its entire memory to see if that data value isstored within the CAM. If the data value is found, the CAM returns alist of one or more storage addresses where the data value was found. Insome architectures, a CAM may return the data value or other associatedpieces of data. Thus, a CAM may be considered the hardware embodiment ofwhat in software terms would be called an associative array.

Thus, in an exemplary embodiment, wake-and-go array 1022 may comprise aCAM and associated logic that will be triggered if a transaction appearson bus 1020 that modifies an address stored in the CAM. A transactionthat modifies a value at a target address may be referred to as a“kill”; thus, wake-and-go array 1022 may be said to be “snooping kills.”In this exemplary embodiment, the data values stored in the CAM are thetarget addresses at which threads are waiting for something to bewritten. The address at which a data value, a given target address, isstored is referred to herein as the storage address. Each storageaddress may refer to a thread that is asleep and waiting for an event.Wake-and-go array 1022 may store multiple instances of the same targetaddress, each instance being associated with a different thread waitingfor an event at that target address. Thus, when wake-and-go array 1022snoops a kill at a given target address, wake-and-go array 1022 mayreturn one or more storage addresses that are associated with one ormore sleeping threads.

FIGS. 11A and 11B illustrate a series of instructions that are aprogramming idiom for wake-and-go in accordance with an illustrativeembodiment. With reference to FIG. 11A, the instruction sequenceincludes load (LD), compare (CMP), and branch (BC) commands thatrepresent a programming idiom that indicate that the thread is waitingfor data to be written to a particular target address. The load command(LD) loads a data value to general purpose register GPR D from theaddress in general purpose register GPR A. The compare command (CMP)then compares the value loaded into general purpose register GPR D witha value already stored in general purpose register GPR E. If the comparecommand results in a match, then the branch command (BC) branches toinstruction address IA.

The wake-and-go mechanism may recognize the poll operation idiom. Whenthe wake-and-go mechanism recognizes such a programming idiom, thewake-and-go mechanism may store the target address from GPR A in thewake-and-go array, where the event the thread is waiting for isassociated with the target address. After updating the wake-and-go arraywith the target address, the wake-and-go mechanism may put the thread tosleep.

With reference now to FIG. 11B, thread 1110 may have a plurality ofprogramming idioms. The wake-and-go mechanism may look ahead withinthread 1110 and load wake-and-go array 1122 with the target address andother information, if any. Therefore, when thread 1110 reaches eachprogramming idiom while executing, the wake-and-go array 1122 willalready be loaded with the target address, and thread 1110 may simply goto sleep until wake-and-go array snoops the target address on the SMPfabric.

The wake-and-go mechanism may perform a look-ahead polling operation foreach programming idiom. In the depicted example, idioms A, B, C, and Dfail. In those cases, the wake-and-go mechanism may update wake-and-goarray 1122. In this example, idiom E passes; therefore, there is no needto update wake-and-go array 1122, because there is no need to put thethread to sleep when idiom E executes.

In one exemplary embodiment, the wake-and-go mechanism may updatewake-and-go array 1122 only if all of the look-ahead polling operationsfail. If at least one look-ahead polling operation passes, then thewake-and-go mechanism may consider each idiom as it occurs duringexecution.

FIGS. 12A and 12B are block diagrams illustrating operation of ahardware wake-and-go mechanism in accordance with an illustrativeembodiment. With particular reference to FIG. 12A, thread 1210 runs in aprocessor (not shown) and performs some work. Thread 1210 executes aseries of instructions that are a programming idiom for wake-and-go. Thewake-and-go mechanism may recognize the poll operation idiom. When thewake-and-go mechanism recognizes such a programming idiom, thewake-and-go mechanism may store the target address A₂ in wake-and-goarray 1222, where the event the thread is waiting for is associated withthe target address, and stores thread state information for thread 1210in thread state storage 1212. After updating wake-and-go array 1222 withthe target address A₂, the wake-and-go mechanism may put the thread 1210to sleep.

When a transaction appears on SMP fabric 1220 with an address thatmatches the target address A₂, array 1222 returns the storage addressthat is associated with thread 1210. The wake-and-go mechanism thenwakes thread 1210 by retrieving the thread state information from threadstate storage 1212 and placing the thread in the run queue for theprocessor. Thread 1210 may then perform a compare-and-branch operationto determine whether the value written to the target address representsthe event for which thread 1210 is waiting. In the depicted example, thevalue written to the target address does not represent the event forwhich thread 1210 is waiting; therefore, thread 1210 goes back to sleep.

Turning to FIG. 12B, thread 1210 runs in a processor (not shown) andperforms some work. Thread 1210 executes a series of instructions thatare a programming idiom for wake-and-go. The wake-and-go mechanism mayrecognize the poll operation idiom. When the wake-and-go mechanismrecognizes such a programming idiom, the wake-and-go mechanism may storethe target address A₂ in wake-and-go array 1222, where the event thethread is waiting for is associated with the target address, and storesthread state information for thread 1210 in thread state storage 1212.After updating wake-and-go array 1222 with the target address A₂, thewake-and-go mechanism may put the thread 1210 to sleep.

When a transaction appears on SMP fabric 1220 with an address thatmatches the target address A₂, array 1222 returns the storage addressthat is associated with thread 1210. The wake-and-go mechanism thenwakes thread 1210 by retrieving the thread state information from threadstate storage 1212 and placing the thread in the run queue for theprocessor. Thread 1210 may then perform a compare-and-branch operationto determine whether the value written to the target address representsthe event for which thread 1210 is waiting. In the depicted example, thevalue written to the target address does represent the event for whichthread 1210 is waiting; therefore, thread 1210 updates the array toremove the target address from array 1222, and performs more work.

FIGS. 13A and 13B are flowcharts illustrating operation of a hardwarewake-and-go mechanism in accordance with the illustrative embodiments.Operation begins when a thread first initializes or when a thread wakesafter sleeping. The operating system starts a thread (block 1302) byinitializing the thread and placing the thread in the run queue for aprocessor. The thread then performs work (block 1304). The operatingsystem determines whether the thread has completed (block 1306). If thethread completes, then operation ends.

If the end of the thread is not reached in block 1306, the processordetermines whether the next instructions comprise a wake-and-go idiom,such as a polling operation, for example (block 1308). A wake-and-goidiom may comprise a series of instructions, such as a load, compare,and branch sequence, for example. If the next instructions doe notcomprise a wake-and-go idiom, the wake-and-go mechanism returns to block1304 to perform more work.

If the next instructions do comprise a wake-and-go idiom in block 1308,the wake-and-go mechanism determines whether to put the thread to sleep(block 1310). The wake-and-go mechanism may keep the thread active inthe processor if the processor is underutilized, for instance; however,the wake-and-go mechanism may put the thread to sleep if there are otherthreads waiting to be run on the processor. If the wake-and-go mechanismdetermines that the thread is to remain active, operation returns toblock 1304 to perform more work, in which case the thread may simplywait for the event.

If the wake-and-go mechanism determines that the thread is to be put tosleep in block 1310, then the wake-and-go mechanism updates the arraywith a target address associated with an event for which the thread iswaiting (block 1312). The update to the wake-and-go array may be made bythe thread through a specialized processor instruction, the operatingsystem, or a background sleeper thread. Next, the wake-and-go mechanismthen saves the state of the thread (block 1314) and puts the thread tosleep (block 1316). Thereafter, operation proceeds to FIG. 13B where thewake-and-go mechanism monitors for an event.

With reference now to FIG. 13B, the wake-and-go mechanism, which mayinclude a wake-and-go array, such as a content addressable memory, andassociated logic, snoops for a kill from the symmetric multiprocessing(SMP) fabric (block 1318). A kill occurs when a transaction appears onthe SMP fabric that modifies the target address associated with theevent for which a thread is waiting. The wake-and-go mechanism, theoperating system, the thread, or other software then performs a compare(block 1320) and determines whether the value being written to thetarget address represents the event for which the thread is waiting(block 1322). If the kill corresponds to the event for which the threadis waiting, then the wake-and-go mechanism updates the array (block1324) to remove the target address from the wake-and-go array.Thereafter, operation returns to block 1302 in FIG. 13A where theoperating system restarts the thread.

If the kill does not correspond to the event for which the thread iswaiting in block 1322, then operation returns to block 1318 to snoop akill from the SMP fabric. In FIG. 13B, the wake-and-go mechanism may bea combination of hardware within the processor, logic associated withthe wake-and-go array, which may be a CAM as described above, andsoftware within the operating system, software within a backgroundsleeper thread. In other embodiments, the wake-and-go mechanism may beother software or hardware, such as a dedicated wake-and-go engine, asdescribed in further detail below.

Look-Ahead Polling

FIGS. 14A and 14B are block diagrams illustrating operation of awake-and-go engine with look-ahead in accordance with an illustrativeembodiment. With particular reference to FIG. 14A, thread 1410 runs in aprocessor (not shown) and performs some work. Thread 1410 executes aseries of instructions that are a programming idiom for wake-and-go. Thewake-and-go mechanism may recognize the poll operation idiom. When thewake-and-go mechanism recognizes such a programming idiom, thewake-and-go mechanism may store the target address A₂ in wake-and-goarray 1422, where the event the thread is waiting for is associated withthe target address, and stores thread state information for thread 1410in thread state storage 1412. After updating wake-and-go array 1422 withthe target address A₂, the wake-and-go mechanism may put the thread 1410to sleep.

When a transaction appears on SMP fabric 1420 with an address thatmatches the target address A₂, array 1422 returns the storage addressthat is associated with thread 1410. The wake-and-go mechanism thenwakes thread 1410 by retrieving the thread state information from threadstate storage 1412 and placing the thread in the run queue for theprocessor. Thread 1410 may then perform a compare-and-branch operationto determine whether the value written to the target address representsthe event for which thread 1410 is waiting. In the depicted example, thevalue written to the target address does not represent the event forwhich thread 1410 is waiting; therefore, thread 1410 goes back to sleep.

Turning to FIG. 14B, thread 1410 runs in a processor (not shown) andperforms some work. Thread 1410 executes a series of instructions thatare a programming idiom for wake-and-go. The wake-and-go mechanism mayrecognize the poll operation idiom. When the wake-and-go mechanismrecognizes such a programming idiom, the wake-and-go mechanism may storethe target address A₂ in wake-and-go array 1422, where the event thethread is waiting for is associated with the target address, and storesthread state information for thread 1410 in thread state storage 1412.After updating wake-and-go array 1422 with the target address A₂, thewake-and-go mechanism may put the thread 1410 to sleep.

When a transaction appears on SMP fabric 1420 with an address thatmatches the target address A₂, array 1422 returns the storage addressthat is associated with thread 1410. The wake-and-go mechanism thenwakes thread 1410 by retrieving the thread state information from threadstate storage 1412 and placing the thread in the run queue for theprocessor. Thread 1410 may then perform a compare-and-branch operationto determine whether the value written to the target address representsthe event for which thread 1410 is waiting. In the depicted example, thevalue written to the target address does represent the event for whichthread 1410 is waiting; therefore, thread 1410 updates the array toremove the target address from array 1422, and performs more work.

FIG. 15 is a flowchart illustrating a look-ahead polling operation of awake-and-go look-ahead engine in accordance with an illustrativeembodiment. Operation begins, and the wake-and-go look-ahead engineexamines the thread for programming idioms (block 1502). Then, thewake-and-go look-ahead engine determines whether it has reached the endof the thread (block 1504). If the wake-and-go look-ahead engine hasreached the end of the thread, operation ends.

If the wake-and-go look-ahead engine has not reached the end of thethread in block 1504, the wake-and-go look-ahead engine determineswhether the thread comprises at least one wake-and-go programming idiomthat indicates that the thread is waiting for a data value to be writtento a particular target address (block 1506). If the thread does notcomprise a wake-and-go programming idiom, operation ends.

If the thread does comprise at least one wake-and-go programming idiomin block 1506, then the wake-and-go look-ahead engine performs load andcompare operations for the at least one wake-and-go programming idiom(block 1508). Thereafter, the wake-and-go look-ahead engine determineswhether all of the load and compare operations fail (block 1510). If allof the look-ahead polling operations fail, then the wake-and-golook-ahead engine updates the wake-and-go array for the at least oneprogramming idiom (block 1512), and operation ends. If at least onelook-ahead polling operation succeeds, then operation ends withoutupdating the wake-and-go array. In an alternative embodiment, thelook-ahead engine may set up the wake-and-go array without performinglook-ahead polling.

Speculative Execution

FIG. 16 is a block diagram illustrating operation of a wake-and-gomechanism with speculative execution in accordance with an illustrativeembodiment. Thread 1610 runs in a processor (not shown) and performssome work. Thread 1610 also includes a series of instructions that are aprogramming idiom for wake-and-go (idiom A), along with idioms B, C, D,and E from FIG. 11B.

Look-ahead wake-and-go engine 1620 analyzes the instructions in thread410 ahead of execution. Look-ahead wake-and-go engine 1620 may recognizethe poll operation idioms and perform look-ahead polling operations foreach idiom. If the look-ahead polling operation fails, the look-aheadwake-and-go engine 1620 populates wake-and-go array 1622 with the targetaddress. In the depicted example from FIG. 11B, idioms A-D fail;therefore, look-ahead wake-and-go engine 1620 populates wake-and-goarray 1622 with addresses A₁-A₄, which are the target addresses foridioms A-D.

If a look-ahead polling operation succeeds, look-ahead wake-and-goengine 1620 may record an instruction address for the correspondingidiom so that the wake-and-go mechanism may have thread 1610 performspeculative execution at a time when thread 1610 is waiting for anevent. During execution, when the wake-and-go mechanism recognizes aprogramming idiom, the wake-and-go mechanism may store the thread statein thread state storage 1612. Instead of putting thread 1610 to sleep,the wake-and-go mechanism may perform speculative execution.

When a transaction appears on SMP fabric 1620 with an address thatmatches the target address A₁, array 1622 returns the storage addressthat is associated with thread 1610 to the wake-and-go mechanism. Thewake-and-go mechanism then returns thread 1610 to the state at whichidiom A was encountered by retrieving the thread state information fromthread state storage 1612. Thread 1610 may then continue work from thepoint of idiom A.

FIG. 17 is a flowchart illustrating operation of a look-aheadwake-and-go mechanism with speculative execution in accordance with anillustrative embodiment. Operation begins, and the wake-and-golook-ahead engine examines the thread for programming idioms (block1702). Then, the wake-and-go look-ahead engine determines whether it hasreached the end of the thread (block 1704). If the wake-and-golook-ahead engine has reached the end of the thread, operation ends.

If the wake-and-go look-ahead engine has not reached the end of thethread in block 1704, the wake-and-go look-ahead engine determineswhether next sequence of instructions comprises a wake-and-goprogramming idiom that indicates that the thread is waiting for a datavalue to be written to a particular target address (block 1706). If thenext sequence of instructions does not comprise a wake-and-goprogramming idiom, operation returns to block 502 to examine the nextsequence of instructions in the thread. A wake-and-go programming idiommay comprise a polling idiom, as described with reference to FIG. 11A.

If the next sequence of instructions does comprise a wake-and-goprogramming idiom in block 1706, then the wake-and-go look-ahead engineperforms load and compare operations for the wake-and-go programmingidiom (block 1708). Thereafter, the wake-and-go look-ahead enginedetermines whether the load and compare operation passes (block 1710).If the look-ahead polling operation fails, then the wake-and-golook-ahead engine updates the wake-and-go array for the programmingidiom (block 1712), and operation returns to block 1702 to examine thenext sequence of instructions in the thread. If the look-ahead pollingoperation passes, then the look-ahead wake-and-go engine records aninstruction address for the successful programming idiom to be used forspeculative execution later (block 1714). Thereafter, operation ends.

FIGS. 18A and 18B are flowcharts illustrating operation of a wake-and-gomechanism with speculative execution during execution of a thread inaccordance with an illustrative embodiment. With reference now to FIG.18A, operation begins when a thread first initializes or when a threadwakes after sleeping. The operating system starts a thread (block 1802)by initializing the thread and placing the thread in the run queue for aprocessor. The thread then performs work (block 1804). The operatingsystem determines whether the thread has completed (block 1806). If thethread completes, then operation ends.

If the end of the thread is not reached in block 1806, the processordetermines whether the next instructions comprise a wake-and-go idiom,such as a polling operation, for example (block 1808). A wake-and-goidiom may comprise a series of instructions, such as a load, compare,and branch sequence, for example. If the next instructions do notcomprise a wake-and-go idiom, the wake-and-go mechanism returns to block1804 to perform more work.

If the next instructions do comprise a wake-and-go idiom in block 1808,the wake-and-go mechanism saves the state of the thread (block 1810).Then, the wake-and-go mechanism determines whether to performspeculative execution (block 1812). The wake-and-go mechanism may makethis determination by determining whether the look-ahead wake-and-goengine previously performed a successful look-ahead polling operationand recorded an instruction address.

If the wake-and-go mechanism determines that the processor cannotperform speculative execution, the wake-and-go mechanism puts the threadto sleep. Thereafter, operation proceeds to FIG. 18B where thewake-and-go mechanism monitors for an event.

If the wake-and-go mechanism determines that the processor can performspeculative execution from a successful polling idiom, the wake-and-gomechanism begins performing speculative execution from the successfullypolled idiom (block 616). Thereafter, operation proceeds to FIG. 18Bwhere the wake-and-go mechanism monitors for an event.

With reference now to FIG. 18B, the wake-and-go mechanism, which mayinclude a wake-and-go array, such as a content addressable memory, andassociated logic, snoops for a kill from the symmetric multiprocessing(SMP) fabric (block 1818). A kill occurs when a transaction appears onthe SMP fabric that modifies the target address associated with theevent for which a thread is waiting. The wake-and-go mechanism, theoperating system, the thread, or other software then performs a compare(block 1820) and determines whether the value being written to thetarget address represents the event for which the thread is waiting(block 1822). If the kill corresponds to the event for which the threadis waiting, then the wake-and-go mechanism updates the array (block1824) to remove the target address from the wake-and-go array.Thereafter, operation returns to block 1804 in FIG. 18A where theprocessor performs more work.

If the kill does not correspond to the event for which the thread iswaiting in block 1822, then operation returns to block 1818 to snoop akill from the SMP fabric. In FIG. 18B, the wake-and-go mechanism may bea combination of hardware within the processor, logic associated withthe wake-and-go array, such as a CAM, and software within the operatingsystem, software within a background sleeper thread, or other hardware.

Data Monitoring

Returning to FIG. 10, the instructions may comprise a get-and-comparesequence, for example. Wake-and-go mechanism 1008 within processor 1000may recognize the get-and-compare sequence as a programming idiom thatindicates that thread 1002 is waiting for data at a particular targetaddress. When wake-and-go mechanism 1008 recognizes such a programmingidiom, wake-and-go mechanism 1008 may store the target address, the datathread 1002 is waiting for, and a comparison type in wake-and-go array1022, where the event the thread is waiting for is associated with thetarget address. After updating wake-and-go array 1022 with the targetaddress, wake-and-go mechanism 1008 may put thread 1002 to sleep.

The get-and-compare sequence may load a data value from a targetaddress, perform a compare operation based on an expected data value,and branch if the compare operation matches. Thus, the get-and-comparesequence had three basic elements: an address, an expected data value,and a comparison type. The comparison type may be, for example, equal to(=), less than (<), greater than (>), less than or equal to (≦), orgreater than or equal to (≧). Thus, wake-and-go mechanism 1008 may storethe address, data value, and comparison value in wake-and-go array 1022.

Thread 1002 may alternatively include specialized processorinstructions, operating system calls, or application programminginterface (API) calls that instruct wake-and-go mechanism 1008 topopulate wake-and-go array 1022 with a given address, data value, andcomparison type.

Wake-and-go mechanism 1008 also may save the state of thread 1002 inthread state storage 1034, which may be allocated from memory 1032 ormay be a hardware private array within the processor (not shown) orpervasive logic (not shown). When a thread is put to sleep, i.e.,removed from the run queue of a processor, the operating system muststore sufficient information on its operating state such that when thethread is again scheduled to run on the processor, the thread can resumeoperation from an identical position. This state information is sometimereferred to as the thread's “context.” The state information mayinclude, for example, address space, stack space, virtual address space,program counter, instruction register, program status word, and thelike.

If a transaction appears on bus 1020 that modifies a value at an addresswhere the value satisfies the comparison type in wake-and-go array 1022,then wake-and-go mechanism 1008 may wake thread 1002. Wake-and-go array1022 may have associated logic that recognizes the target address on bus1020 and performs the comparison based on the value being written, theexpected value stored in wake-and-go array 1022, and the comparison typestored in wake-and-go array 1022. Wake-and-go mechanism 1008 may wakethread 1002 by recovering the state of thread 1002 from thread statestorage 1034. Thus, thread 1002 only wakes if there is a transactionthat modifies the target address with a value that satisfies thecomparison type and expected value.

Thus, in an exemplary embodiment, wake-and-go array 1022 may comprise aCAM and associated logic that will be triggered if a transaction appearson bus 1020 that modifies an address stored in the CAM. A transactionthat modifies a value at a target address may be referred to as a“kill”; thus, wake-and-go array 1022 may be said to be “snooping kills.”In this exemplary embodiment, the data values stored in the CAM are thetarget addresses at which threads are waiting for something to bewritten, an expected value, and a comparison type. The address at whicha data value, a given target address, is stored is referred to herein asthe storage address.

Each storage address may refer to a thread that is asleep and waitingfor an event. Wake-and-go array 1022 may store multiple instances of thesame target address, each instance being associated with a differentthread waiting for an event at that target address. The expected valuesand comparison types may be different. Thus, when wake-and-go array 1022snoops a kill at a given target address, wake-and-go array 1022 mayreturn one or more storage addresses that are associated with one ormore sleeping threads. When wake-and-go array 1022 snoops a kill at thegiven target address, wake-and-go array 1022 may also return theexpected value and comparison type to associated logic that performs thecomparison. If the comparison matches, then the associated logic mayreturn a storage address to wake-and-go mechanism 1008 to wake thecorresponding thread.

FIG. 19 is a block diagram illustrating data monitoring in a multipleprocessor system in accordance with an illustrative embodiment.Processors 1902-1908 connect to bus 1920. Each one of processors1902-1908 may have a wake-and-go mechanism, such as wake-and-gomechanism 1008 in FIG. 10, and a wake-and-go array, such as wake-and-goarray 1022 in FIG. 10. A device (not shown) may modify a data value at atarget address through input/output channel controller (HOC) 1912, whichtransmits the transaction on bus 1920 to memory controller 1914.

The wake-and-go array of each processor 1902-1908 snoops bus 1920. If atransaction appears on bus 1920 that modifies a value at an addresswhere the value satisfies the comparison type in a wake-and-go array,then the wake-and-go mechanism may wake a thread. Each wake-and-go arraymay have associated logic that recognizes the target address on bus 1920and performs the comparison based on the value being written, theexpected value stored in the wake-and-go array, and the comparison typestored in the wake-and-go array. Thus, the wake-and-go mechanism mayonly wake a thread if there is a transaction on bus 1920 that modifiesthe target address with a value that satisfies the comparison type andexpected value.

FIG. 20 is a block diagram illustrating operation of a wake-and-gomechanism in accordance with an illustrative embodiment. Thread 2010runs in a processor (not shown) and performs some work. Thread 2010executes a series of instructions that are a programming idiom forwake-and-go, a specialized processor instruction, an operating systemcall, or an application programming interface (API) call. Thewake-and-go mechanism may recognize the idiom, specialized processorinstruction, operating system call, or API call, hereinafter referred toas a “wake-and-go operation.” When the wake-and-go mechanism recognizessuch a wake-and-go operation, the wake-and-go mechanism may store thetarget address A₂, expected data value D₂, and comparison type T₂ inwake-and-go array 2022, and stores thread state information for thread2010 in thread state storage 2012. After updating wake-and-go array 2022with the target address A₂, expected data value D₂, and comparison typeT₂, the wake-and-go mechanism may put thread 2010 to sleep.

When a transaction appears on SMP fabric 2020 with an address thatmatches the target address A₂, logic associated with wake-and-go array2022 may perform a comparison based on the value being written, theexpected value D₂ and the comparison type T₂. If the comparison is amatch, then the logic associated with wake-and-go array 2022 returns thestorage address that is associated with thread 2010. The wake-and-gomechanism then wakes thread 2010 by retrieving the thread stateinformation from thread state storage 2012 and placing the thread in therun queue for the processor.

Parallel Lock Spinning

Returning to FIG. 10, the instructions may comprise a get-and-comparesequence, for example. In an illustrative embodiment, the instructionsmay comprise a sequence of instructions that indicate that thread 1002is spinning on a lock. A lock is a synchronization mechanism forenforcing limits on access to resources in an environment where thereare multiple threads of execution. Generally, when a thread attempts towrite to a resource, the thread may request a lock on the resource toobtain exclusive access. If another thread already has the lock, thethread may “spin” on the lock, which means repeatedly polling the locklocation until the lock is free. The instructions for spinning on thelock represent an example of a programming idiom.

Wake-and-go mechanism 1008 within processor 1000 may recognize thespinning on lock idiom that indicates that thread 1002 is spinning on alock. When wake-and-go mechanism 1008 recognizes such a programmingidiom, wake-and-go mechanism 1008 may store the target address inwake-and-go array 1022 with a flag to indicate that thread 1002 isspinning on a lock. After updating wake-and-go array 1022 with thetarget address and setting the lock flag, wake-and-go mechanism 1008 mayput thread 1002 to sleep. Thus, wake-and-go mechanism 1008 allowsseveral threads to be spinning on a lock at the same time without usingvaluable processor resources.

If a transaction appears on bus 1020 that modifies a value at an addressin wake-and-go array 1022, then wake-and-go mechanism 1008 may wakethread 1002. Wake-and-go mechanism 1008 may wake thread 1002 byrecovering the state of thread 1002 from thread state storage 1034.Thread 1002 may then determine whether the transaction corresponds tothe event for which the thread was waiting by performing aget-and-compare operation, for instance. If the lock bit is set inwake-and-go array 1022, then it is highly likely that the transaction isfreeing the lock, in which case, wake-and-go mechanism may automaticallywake thread 1002.

FIGS. 21A and 21B are block diagrams illustrating parallel lock spinningusing a wake-and-go mechanism in accordance with an illustrativeembodiment. With particular reference to FIG. 21A, thread 2110 runs in aprocessor (not shown) and performs some work. Thread 2110 executes aseries of instructions that are a programming idiom for spin on lock.The wake-and-go mechanism may recognize the spin on lock operationidiom. When the wake-and-go mechanism recognizes such a programmingidiom, the wake-and-go mechanism may store the target address A₁ inwake-and-go array 2122, set the lock bit 2124, and store thread stateinformation for thread 2110 in thread state storage 2112. After updatingwake-and-go array 2122 with the target address A₁, the wake-and-gomechanism may put the thread 2110 to sleep.

The processor may then run thread 2130, which performs some work. Thewake-and-go mechanism may recognize a spin on lock operation idiom,responsive to which the wake-and-go mechanism stores the target addressA₂ in wake-and-go array 2122, set the lock bit 2124, and store threadstate information for thread 2130 in thread state storage 2112. Afterupdating wake-and-go array 2122 with the target address A₂, thewake-and-go mechanism may put the thread 2130 to sleep.

Turning to FIG. 21B, thread 2140 runs in the processor and performs somework. When a transaction appears on SMP fabric 2120 with an address thatmatches the target address A₁, wake-and-go array 2122 returns thestorage address that is associated with thread 2110. The wake-and-gomechanism then wakes thread 2110 by retrieving the thread stateinformation from thread state storage 2112 and placing the thread in therun queue for the processor, because it is highly likely that thetransaction is freeing the lock. Thread 2110 may update array 2122 toremove the target address. In the depicted example, thread 2110 andthread 2140 run concurrently in the processor. Thus, thread 2110 andthread 2130, and any number of other threads, may be spinning on a lockat the same time. When a lock is freed, the processor may wake thethread, such as thread 2110 in the depicted example, and the remainingthreads may continue “spinning” on the lock without consuming anyprocessor resources.

FIGS. 22A and 22B are flowcharts illustrating parallel lock spinningusing a wake-and-go mechanism in accordance with the illustrativeembodiments. Operation begins when a thread first initializes or when athread wakes after sleeping. The operating system starts a thread (block2202) by initializing the thread and placing the thread in the run queuefor a processor. The thread then performs work (block 2204). Theoperating system determines whether the thread has completed (block2206). If the thread completes, then operation ends.

If the end of the thread is not reached in block 2206, the processordetermines whether the next instructions comprise a spin on lock idiom(block 2208). A spin on lock idiom may comprise a series ofinstructions, such as a load, compare, and branch sequence, for example.If the next instructions do not comprise a spin on lock idiom, thewake-and-go mechanism returns to block 2204 to perform more work.

If the next instructions do comprise a spin on lock idiom in block 2208,the wake-and-go mechanism updates the array with a target addressassociated with an event for which the thread is waiting (block 2210)and sets the lock bit in the wake-and-go array (block 2212). The updateto the wake-and-go array may be made by the thread through a specializedprocessor instruction, the operating system, or a background sleeperthread. Next, the wake-and-go mechanism saves the state of the thread(block 2214) and puts the thread to sleep (block 2216). Thereafter,operation proceeds to FIG. 22B where the wake-and-go mechanism monitorsfor an event.

With reference now to FIG. 22B, the wake-and-go mechanism, which mayinclude a wake-and-go array, such as a content addressable memory (CAM),and associated logic, snoops for a kill from the symmetricmultiprocessing (SMP) fabric (block 2218). A kill occurs when atransaction appears on the SMP fabric that modifies the target addressassociated with the event for which a thread is waiting. The wake-and-gomechanism determines whether the value being written to the targetaddress represents the event for which the thread is waiting (block2220). If the lock bit is set, then it is highly likely that the eventis merely freeing the lock. If the kill corresponds to the event forwhich the thread is waiting, then the wake-and-go mechanism updates thearray (block 2222) to remove the target address from the wake-and-goarray and reloads the thread state for the thread that was spinning onthe lock (block 2224). Thereafter, operation returns to block 2202 inFIG. 22A where the operating system restarts the thread.

If the kill does not correspond to the event for which the thread iswaiting in block 2220, then operation returns to block 2218 to snoop akill from the SMP fabric. In FIG. 22B, the wake-and-go mechanism may bea combination of hardware within the processor, logic associated withthe wake-and-go array, such as a CAM, and software within the operatingsystem, software within a background sleeper thread, or other hardware.

Central Repository for Wake-and-Go Engine

As stated above with reference to FIG. 10, while the data processingsystem in FIG. 10 shows one processor, more processors may be presentdepending upon the implementation where each processor has a separatewake-and-go array or one wake-and-go array stores target addresses forthreads for multiple processors. In one illustrative embodiment, onewake-and-go engine stores entries in a central repository wake-and-goarray for all threads and multiple processors.

FIG. 23 is a block diagram illustrating a wake-and-go engine with acentral repository wake-and-go array in a multiple processor system inaccordance with an illustrative embodiment. Processors 2302-2308 connectto bus 2320. A device (not shown) may modify a data value at a targetaddress through input/output channel controller (HOC) 2312, whichtransmits the transaction on bus 2320 to memory controller 2314.Wake-and-go engine 2350 performs look-ahead to identify wake-and-goprogramming idioms in the instruction streams of threads running onprocessors 2302-2308. If wake-and-go engine 2350 recognizes awake-and-go programming idiom, wake-and-go engine 2350 records an entryin central repository wake-and-go array 2352.

Wake-and-go engine 2350 snoops bus 2320. If a transaction appears on bus2320 that modifies a value at an address where the value satisfies thecomparison type in a wake-and-go array, then the wake-and-go engine 2350may wake a thread. Wake-and-go engine 2350 may have associated logicthat recognizes the target address on bus 2320 and performs thecomparison based on the value being written, the expected value storedin the wake-and-go array, and the comparison type stored in centralrepository wake-and-go array 2352. Thus, wake-and-go engine 2350 mayonly wake a thread if there is a transaction on bus 2320 that modifiesthe target address with a value that satisfies the comparison type andexpected value.

FIG. 24 illustrates a central repository wake-and-go-array in accordancewith an illustrative embodiment. Each entry in central repositorywake-and-go array 2400 may include thread identification (ID) 2402,central processing unit (CPU) ID 2404, the target address 2406, theexpected data 2408, a comparison type 2410, a lock bit 2412, a priority2414, and a thread state pointer 2416, which is the address at which thethread state information is stored.

The wake-and-go engine 2350 may use the thread ID 2402 to identify thethread and the CPU ID 2404 to identify the processor. Wake-and-go engine2350 may then place the thread in the run queue for the processoridentified by CPU ID 2404. Wake-and-go engine 2350 may also use threadstate pointer 2416 to load thread state information, which is used towake the thread to the proper state.

Programming Idiom Accelerator

In a sense, a wake-and-go mechanism, such as look-ahead wake-and-goengine 2350, is a programming idiom accelerator. A programming idiom isa sequence of programming instructions that occurs often and isrecognizable as a sequence of instructions. In the examples describedabove, an instruction sequence that includes load (LD), compare (CMP),and branch (BC) commands represents a programming idiom that indicatesthat the thread is waiting for data to be written to a particular targetaddress. Wake-and-go engine 2350 recognizes this idiom as a wake-and-goidiom and accelerates the wake-and-go process accordingly, as describedabove. Other examples of programming idioms may include spinning on alock or traversing a linked list.

FIG. 25 is a block diagram illustrating a programming idiom acceleratorin accordance with an illustrative embodiment. Processors 2502-2508connect to bus 2520. A processor, such as processor 2502 for example,may fetch instructions from memory via memory controller 2514. Asprocessor 2502 fetches instructions, programming idiom accelerator 2550may look ahead to determine whether a programming idiom is coming up inthe instruction stream. If programming idiom accelerator 2550 recognizesa programming idiom, programming idiom accelerator 2550 performs anaction to accelerate execution of the programming idiom. In the case ofa wake-and-go programming idiom, programming idiom accelerator 2550 mayrecord an entry in a wake-and-go array, for example.

As another example, if programming idiom accelerator 2550 accelerateslock spinning programming idioms, programming idiom accelerator 2550 mayobtain the lock for the processor, if the lock is available, thus makingthe lock spinning programming sequence of instructions unnecessary.Programming idiom accelerator 2550 may accelerate any known or commonsequence of instructions or future sequences of instructions. Althoughnot shown in FIG. 25, a data processing system may include multipleprogramming idiom accelerators that accelerate various programmingidioms. Alternatively, programming idiom accelerator 2550 may recognizeand accelerator multiple known programming idioms. In one exemplaryembodiment, each processor 2502-2508 may have programming idiomaccelerators within the processor itself.

As stated above with respect to the wake-and-go engine, programmingidiom accelerator 2550 may be a hardware device within the dataprocessing system. In an alternative embodiment, programming idiomaccelerator 2550 may be a hardware component within each processor2502-2508. In another embodiment, programming idiom accelerator 2550 maybe software within an operating system running on one or more ofprocessors 2502-2508. Thus, in various implementations or embodiments,programming idiom accelerator 2550 may be software, such as a backgroundsleeper thread or part of an operating system, hardware, or acombination of hardware and software.

More particularly, examples of programming idioms may include locallocking, global locking, local barriers, global barriers, and globalcollectives. The programming idiom accelerator my perform operations toaccelerate execution of the above programming idioms.

In one embodiment, the programming language may include hintinstructions that may notify programming accelerator 2550 that aprogramming idiom is coming. FIG. 26 is a series of instructions thatare a programming idiom with programming language exposure in accordancewith an illustrative embodiment. In the example depicted in FIG. 26, theinstruction stream includes programming idiom 2602, which in this caseis an instruction sequence that includes load (LD), compare (CMP), andbranch (BC) commands that indicate that the thread is waiting for datato be written to a particular target address.

Idiom begin hint 2604 exposes the programming idiom to the programmingidiom accelerator. Thus, the programming idiom accelerator need notperform pattern matching or other forms of analysis to recognize asequence of instructions. Rather, the programmer may insert idiom hintinstructions, such as idiom begin hint 2604, to expose the idiom 2602 tothe programming idiom accelerator. Similarly, idiom end hint 2606 maymark the end of the programming idiom; however, idiom end hint 2606 maybe unnecessary if the programming idiom accelerator is capable ofidentifying the sequence of instructions as a recognized programmingidiom.

In an alternative embodiment, a compiler may recognize programmingidioms and expose the programming idioms to the programming idiomaccelerator. FIG. 27 is a block diagram illustrating a compiler thatexposes programming idioms in accordance with an illustrativeembodiment. Compiler 2710 receives high level program code 2702 andcompiles the high level instructions into machine instructions to beexecuted by a processor. Compiler 2710 may be software running on a dataprocessing system, such as data processing system 100 in FIG. 1, forexample.

Compiler 2710 includes programming idiom exposing module 2712, whichparses high level program code 2702 and identifies sequences ofinstructions that are recognized programming idioms. Compiler 2710 thencompiles the high level program code 2702 into machine instructions andinserts hint instructions to expose the programming idioms. Theresulting compiled code is machine code with programming idioms exposed2714. As machine code 2714 is fetched for execution by a processor, oneor more programming idiom accelerators may see a programming idiomcoming up and perform an action to accelerate execution.

FIG. 28 is a flowchart illustrating operation of a compiler exposingprogramming idioms in accordance with an illustrative embodiment.Operation begins and the compiler receives high level program code tocompile into machine code (block 2802). The compiler considers asequence of code (block 2804) and determines whether the sequence ofcode includes a recognized programming idiom (block 2806).

If the sequence of code includes a recognized programming idiom, thecompiler inserts one or more instructions to expose the programmingidiom to the programming idiom accelerator (block 2808). The compilercompiles the sequence of code (block 2810). If the sequence of code doesnot include a recognized programming idiom in block 2806, the compilerproceeds to block 2810 to compile the sequence of code.

After compiling the sequence of code in block 2810, the compilerdetermines if the end of the high level program code is reached (block2812). If the end of the program code is not reached, operation returnsto block 2804 to consider the next sequence of high level programinstructions. If the end of the program code is reached in block 2812,then operation ends.

The compiler may recognize one or more programming idioms from a set ofpredetermined programming idioms. The set of predetermined programmingidioms may correspond to a set of programming idiom accelerators thatare known to be supported in the target machine. For example, if thetarget data processing system has a wake-and-go engine and a linked listacceleration engine, then the compiler may provide hints for these twoprogramming idioms. The hint instructions may be such that they areignored by a processor or data processing system that does not supportprogramming idiom accelerators.

Remote Update

One common programming idiom is a remote update. Frequently, a threadincludes a sequence of instructions that attempts to get a socket toread some data from a remote node, perform some operation or a morecomplex series of operations on the data, and write a result back to theremote node. In accordance with an illustrative embodiment, aprogramming idiom accelerator recognizes this programming idiom as aremote update and performs some action to accelerate execution of theidiom. More particularly, the programming idiom may send the remoteupdate to the remote node to have the remote update performed remotely.

FIG. 29 is a block diagram illustrating a data processing system with aremote update programming idiom accelerator in accordance with anillustrative embodiment. Data processing system 2910 connects to dataprocessing system 2940 via network 2902. In data processing system 2910,processors 2912, 2914, and 2916 connect to bus 2930. Memory controller2934 and network interface controller 2936 also connect to bus 2930.Processor 2912 executes operating system 2922, processor 2914 runsoperating system 2924, and processor 2916 runs operating system 2926.Operating systems 2922, 2924, and 2926 run on top of virtualizationlayer 2920.

Virtualization layer 2920 virtualizes resources, such as processorresources, memory resources, hardware resources, and the like. From theperspective of virtualization layer 2920, a (virtual) processor may be awhole physical processor, two or more physical processors, a processorcore, two or more processor cores, a thread within a multi-threadedprocessor or core, or a time slice of a processor or core. Therefore, inthe example depicted in FIG. 29, processors 2912, 2914, 2916 may bevirtual processors from the perspective of virtualization layer 2920.Virtualization layer 2920 may allow several operating systems to run ondata processing system 2910 by creating an operating system instance,assigning a virtual processor and other resources to the operatingsystem. Virtualization layer 2920 may also de-allocate and reallocateresources dynamically and perform workload balancing in a workloadmanager (not shown).

In data processing system 2940, processors 2942, 2944, and 2946 connectto bus 2960. Memory controller 2964 and network interface controller2966 also connect to bus 2960. Processor 2942 executes operating system2952, processor 2944 runs operating system 2954, and processor 2946 runsoperating system 2956. Operating systems 2952, 2954, and 2956 run on topof virtualization layer 2950. While the example depicted in FIG. 29shows three processors in data processing system 2910 and dataprocessing system 2940, each data processing system may include more orfewer processors depending upon the implementation or the allocation ofprocessing resources by virtualization layers 2920 and 2950.Furthermore, a person of ordinary skill in the art will recognize thatmore than two data processing systems may be connected via network 2902.A person of ordinary skill in the art may recognize that othermodifications to the example depicted in FIG. 29 may be made withoutdeparting from the spirit and scope of the present invention.

In the depicted example, data processing system 2910 and data processingsystem 2940 may communicate over the Internet with network 2902representing a worldwide collection of networks and gateways that usethe Transmission Control Protocol/Internet Protocol (TCP/IP) suite ofprotocols to communicate with one another. At the heart of the Internetis a backbone of high-speed data communication lines between major nodesor host computers, consisting of thousands of commercial, governmental,educational and other computer systems that route data and messages. Ofcourse, data processing systems 2910, 2940 may also be implemented toinclude a number of different types of networks, such as for example, anintranet, a local area network (LAN), a wide area network (WAN), or thelike. FIG. 29 is intended as an example, not as an architecturallimitation for different embodiments of the present invention, andtherefore, the particular elements shown in FIG. 29 should not beconsidered limiting with regard to the environments in which theillustrative embodiments of the present invention may be implemented.

Operating systems 2922-2926 and 2952-2956 coordinate and provide controlof various components within data processing systems 2910, 2940. As aclient, the operating system may be a commercially available operatingsystem such as Microsoft® Windows® XP (Microsoft and Windows aretrademarks of Microsoft Corporation in the United States, othercountries, or both). An object-oriented programming system, such as theJava™ programming system, may run in conjunction with the operatingsystem and provides calls to the operating system from Java™ programs orapplications executing on data processing systems 2910, 2940 (Java is atrademark of Sun Microsystems, Inc. in the United States, othercountries, or both).

As a server, data processing system 2910 or data processing system 2940may be, for example, an IBM® eServer™ System p® computer system, runningthe Advanced Interactive Executive (AIX®) operating system or the LINUX®operating system (eServer, System p, and AIX are trademarks ofInternational Business Machines Corporation in the United States, othercountries, or both while LINUX is a trademark of Linus Torvalds in theUnited States, other countries, or both). Data processing system 2910 ordata processing system 2940 may be a symmetric multiprocessor (SMP)system. Alternatively, a single processor system may be employed.

Instructions for the operating system, the object-oriented programmingsystem, and applications or programs are located on storage devices,such as a hard disk drive (not shown), and may be loaded into memory forexecution by processors 2912-2916 or processors 2942-2946. The processesof the illustrative embodiments may be performed by processing units2912-2916 and 2942-2946 using computer usable program code, which may belocated in a memory.

Bus 2930 or bus 2960 as shown in FIG. 29, may be comprised of one ormore buses. Of course, the bus system may be implemented using any typeof communication fabric or architecture that provides for a transfer ofdata between different components or devices attached to the fabric orarchitecture. A communication unit, such network interface controller2936 or network interface controller 2966 of FIG. 29, may include one ormore devices used to transmit and receive data.

A processor, such as processor 2912 for example, may fetch instructionsfrom memory via memory controller 2934. As processor 2912 fetchesinstructions, programming idiom accelerator 2932 may look ahead todetermine whether a programming idiom is coming up in the instructionstream. If programming idiom accelerator 2932 recognizes a programmingidiom, programming idiom accelerator 2932 performs an action toaccelerate execution of the programming idiom.

A thread running in processor 2912, for example, may access data inlocal memory through the operating system 2922, virtualization layer2920, bus 2930, and memory controller 2934. For a remote access, athread running in processor 2914, for example, may access data in memoryat data processing system 2940 through operating system 2924,virtualization layer 2920, bus 2930, network interface controller 2936,network 2902, network interface controller 2966 in data processingsystem 2940, bus 2960, and memory controller 2964. In order to perform aremote access, the thread running in processor 2914 must attempt to geta socket. A socket is a method of directing data to the appropriateapplication in a TCP/IP network. The combination of the IP address ofthe station and a port number make up a socket. Thus, when a thread getsa socket, the thread must traverse software, network, and physicallayers, and address spaces, which is a high-overhead process that may beperformed repeatedly.

In accordance with an illustrative embodiment, programming idiomaccelerator 2932 recognizes a remote update programming idiom andperforms an action to accelerate execution of the programming idiom. Inthe example depicted in FIG. 29, programming idiom accelerator 2932 isillustrated as a special purpose hardware device, such as a dedicatedprocessor, associated with bus 2930. As such, programming idiomaccelerator 2932 resides in the hardware layer, either within eachprocessor 2912-2916 and 2942-2946, within logic for bus 2930, or as adedicated processor connected to bus 2930. In an example embodiment,programming idiom accelerator 2932 may reside in network interfacecontroller 2936 or, alternatively, a host channel adapter or othercommunications adapter. In one example embodiment, programming idiomaccelerator 2932 may be embodied within a host fabric interface (HFI)controller, which is described in co-pending patent application Ser. No.12/342,559, entitled “Management of Process-to-Process CommunicationRequests,” Ser. No. 12/342,616 entitled “Management ofProcess-to-Process Intra-Cluster Communication Requests,”Ser. No.12/342,691 entitled “Management of Process-to-Process Inter-ClusterCommunication Requests,” Ser. No. 12/342,881 entitled “Management ofApplication to I/O Device Communication Requests Between Data ProcessingSystems,” Ser. No. 12/342,834 entitled “Management of Application toApplication Communication Requests Between Data Processing Systems,” allfiled Dec. 23, 2008.

A common “universal” remote update may comprise a read, a simpleoperation, and a write. Examples of such a simple operation may includelogic AND, logic OR, logic XOR, ADD, MULITPLY, atomic increment,compare, and so forth. Thus, a “universal” remote update idiom is acommon and easily recognizable sequence of instructions that read datafrom a remote node, perform a common operation, and return a result tothe remote node. Programming idiom accelerator 2932 may see a sequenceof instructions, such as a read/AND/write, and recognize this sequenceof instructions as one of a set of “universal” remote update idioms.When programming idiom accelerator 2932 identifies a remote updateidiom, programming idiom accelerator 2932 may send the remote update toprogramming idiom accelerator 2962.

Programming idiom accelerator 2962 may in turn perform the remoteupdate, which is local to data processing system 2940. When the updateis completed, programming idiom accelerator 2962 may then return acompletion notification to programming idiom accelerator 2932, whichthen notes that the remote update is complete. As such, programmingidiom accelerator 2932 may anticipate the needs of threads running onprocessors 2912, 2914, 2916, and perform specialized actions to avoidthe inefficient and high-overhead process of getting a socket to performa remote update, particularly when these idioms appear frequently inprogram code.

In one illustrative embodiment, a remote update idiom may include a readoperation to obtain some data from a remote node, a sequence ofinstructions to perform some complex operation on the data, and a writeoperation to return some result to the remote node. Thus, a “complex”remote update is a sequence of instructions that reads data from aremote node, performs a complex, variable-length sequence of operationson the data, and returns result data to the remote node.

Programming idiom accelerator 2932 may see a sequence of instructionsand recognize this sequence of instructions as a “complex” remote updateidiom. When programming idiom accelerator 2932 identifies a complexremote update idiom, programming idiom accelerator 2932 may send theremote update, including the complex sequence of instructions, toprogramming idiom accelerator 2962. Programming idiom accelerator 2962may in turn receive the complex sequence of instructions and perform theremote update, which is local to data processing system 2940. When theupdate is completed, programming idiom accelerator 2962 may then returna completion notification to programming idiom accelerator 2932, whichthen notes that the remote update is complete.

In one illustrative embodiment, programming idiom accelerator 2962 maycomprise special purpose hardware, such as a dedicated processor, forperforming the update operation. Thus, programming idiom accelerator2962 may perform the update and return a completion notification toprogramming idiom accelerator 2932 without significantly affectingoperation of processors 2942, 2944, 2946.

Alternatively, in on illustrative embodiment, programming idiomaccelerator 2962 may submit a request to virtualization layer 2950 forprocessing resources to perform the remote update. In turn,virtualization layer 2950 may assign processing resources, i.e. virtualprocessor, for the purpose of performing the remote update. Programmingidiom accelerator 2962 may then send the complex sequence ofinstructions to the virtual processor, such as processor 2946, whichexecutes the instructions to perform the remote update. The virtualprocessor then notifies programming idiom accelerator 2962 that theremote update is complete, and programming idiom accelerator 2962 maythen send a completion notification back to programming idiomaccelerator 2932.

Programming idiom accelerator 2962 may have an inherent instruction sizethreshold, referred to herein as the dedicated threshold. Alternatively,programming idiom accelerator 2962 may determine the dedicated thresholddynamically based on the demand for remote updates from other nodes. Ifthe instruction size of the complex remote update is below the dedicatedthreshold, programming idiom accelerator 2962 can perform the remoteupdate without requesting processing resources from virtualization layer2950. However, if the instruction size of the complex remote update isabove the dedicated threshold, programming idiom accelerator 2962 mayrequest processing resources from virtualization layer 2950.

Virtualization layer 2950 may provide feedback information toprogramming idiom accelerator 2962. This feedback information mayinclude processor utilization information based on the workload manager(not shown). Using the feedback information, programming idiomaccelerator 2962 may determine a dynamic instruction size, referred toherein as the remote threshold. If the instruction size of the complexremote update is below the remote threshold, virtualization layer 2950can assign processing resources to perform the remote update. Becauseprocessors 2942, 2944, and 2946 are performing other work by executingtheir own threads locally, virtualization layer 2950 will have limitedresources to allocate for a complex remote update. Therefore, the morework data processing system 2940 is doing locally, the fewer processingresources virtualization layer 2950 can allocate for complex remoteupdate, and the lower the remote threshold.

In one embodiment, the thread itself may indicate within an idiom hintthat the complex remote update is to be performed by a virtual processorassigned by the virtualization layer 2950. Thus, the programmer orcompiler may recognize that the processing resources of the remote nodeare the most efficient and effective resources for performing thisparticular sequence of instructions. In this case, the programmer orcompiler may insert idiom hints in the program code to expose the idiomand directly request that the idiom be performed by processor resourcesallocated by the virtualization layer.

It is common practice in clustering data processing, for example, fordata processing systems to provide heartbeat information among thecluster. In accordance with an illustrative embodiment, data processingsystem 2910 and data processing system 2940 may send heartbeatinformation between each other. This heartbeat information may includethe remote threshold. Then, when programming idiom accelerator 2932 indata processing system 2910 encounters a complex remote update idiom,programming idiom accelerator 2932 compares the instruction size of thecomplex remote update to the remote threshold. If the instruction sizeof the complex remote update is greater than the remote threshold, thenprogramming idiom accelerator 2932 cannot send the complex remote updateto data processing system 2940. In this instance, the thread with thecomplex remote update idiom executes normally by getting a socket,reading data from the remote node, performing the complex sequence ofoperations on the data, and writing result data back to the remote node.However, if the instruction size of the complex remote update is lessthan the remote threshold, programming idiom accelerator 2932 can sendthe complex remote update to programming idiom accelerator 2962.

FIGS. 30A and 30B are flowcharts illustrating operation of a remoteupdate in accordance with the illustrative embodiments. With referenceto FIG. 30A, operation begins, and the programming idiom acceleratordetermines whether a remote update idiom occurs in an instruction streamof a thread running locally on the data processing system (block 3002).If the programming idiom accelerator does not encounter a remote updateidiom for updating some data at a remote node, then operation returns toblock 3002. If the programming idiom accelerator encounters a remoteupdate programming idiom in block 3002, the programming idiomaccelerator sends the remote update to the remote node (block 3004).Then, the programming idiom accelerator determines whether a completionnotification is received from the remote node (block 3006). If theprogramming idiom accelerator does not receive a completionnotification, then operation returns to block 3006. If the programmingidiom accelerator receives a completion notification in block 3006,operation returns to block 3002 to determine whether a remote updateidiom occurs in an instruction stream of a thread.

Turning to FIG. 30B, operation begins, and the programming idiomaccelerator determines whether the programming idiom acceleratorreceives a remote update from a remote node (block 3008). If theprogramming idiom accelerator does not receive a remote update from aremote node, then operation returns to block 3008. If the programmingidiom accelerator receives a remote update in block 3008, then the dataprocessing system performs the update locally (block 3010). Thereafter,the programming idiom accelerator notifies the sending node ofcompletion (block 3012), and operation returns to block 3008 todetermine whether the programming idiom accelerator receives a remoteupdate from a remote node.

FIGS. 31A and 31B are flowcharts illustrating operation of a remoteupdate with dynamic instruction size in accordance with the illustrativeembodiments. With reference to FIG. 31A, operation begins, and theprogramming idiom accelerator determines whether it receives remoteinstruction size information from a remote node (block 3102). If theprogramming idiom accelerator receives remote instruction sizeinformation from a remote node, the programming idiom accelerator sets aremote threshold for that node (block 3104). Thereafter, or if theprogramming idiom accelerator does not receive remote instruction sizeinformation in block 3102, the programming idiom accelerator determineswhether it encounters a remote update in the instruction stream of athread running locally on the data processing system (block 3106). Ifthe programming idiom accelerator does not encounter a remote update,operation returns to block 3102 to determine whether the programmingidiom accelerator receives remote instruction size information.

If the programming idiom accelerator encounters a remote update in block3106, the programming idiom accelerator determines whether theinstruction size of the remote update is greater than the remotethreshold for the remote node (block 3108). If the programming idiomaccelerator determines that the instruction size is greater than theremote threshold, then the thread performs the remote update using asocket without acceleration (block 3110), and operation returns to block3102 to determine whether the programming idiom accelerator receivesremote instruction size information.

However, if the programming idiom accelerator determines that theinstruction size of the remote update is not greater than the remotethreshold, then the programming idiom accelerator sends the remoteupdate to the remote node (block 3112. Then, the programming idiomaccelerator determines whether a completion notification is receivedfrom the remote node (block 3114). If the programming idiom acceleratordoes not receive a completion notification, then operation returns toblock 3114. If the programming idiom accelerator receives a completionnotification in block 3114, operation returns to block 3102 to determinewhether the programming idiom accelerator receives remote instructionsize information.

Turning to FIG. 31B, operation begins, and the programming idiomaccelerator determines whether the programming idiom acceleratorreceives processor instruction size information from the virtualizationlayer (block 3116). If the programming idiom accelerator receivesprocessor instruction size information, then the programming idiomaccelerator sets the remote threshold (block 3118) and sends the remotethreshold to the other nodes in heartbeat information (block 3120).Then, the programming idiom accelerator sets the dedicated threshold(block 3122). The dedicated threshold may be constant or may bedetermined dynamically based on a number of remote updates received fromother nodes.

Thereafter, or if the programming idiom accelerator does not receiveprocessor instruction information in block 3116, the programming idiomaccelerator determines whether it receives a remote update from a remotenode (block 3124). If the programming idiom accelerator does not receivea remote update from a remote node, then operation returns to block 3116to determine whether the programming idiom accelerator receivesprocessor instruction size information.

If the programming idiom accelerator receives a remote update from aremote node in block 3124, then the programming idiom acceleratordetermines whether the instruction size of the remote update is greaterthan the dedicated threshold (block 3126). If the programming idiomaccelerator determines that the instruction size of the remote update isnot greater than the dedicated threshold, the programming idiomaccelerator performs the update locally in the dedicated processor ofthe programming idiom accelerator (block 3128). Then, the programmingidiom accelerator notifies the remote node of completion of the remoteupdate (block 3130), and operation returns to block 3116 to determinewhether the programming idiom accelerator receives processor instructionsize information.

If the programming idiom accelerator determines that the instructionsize of the remote update is greater than the dedicated threshold, thenthe programming idiom accelerator requests processor resources from thevirtualization layer (block 3132). The programming idiom acceleratorreceives a virtual processor assignment from the virtualization layer(block 3134). The programming idiom accelerator then performs the updateusing the assigned processor resources (block 3136). Thereafter,operation proceeds to block 3130 where the programming idiom acceleratornotifies the remote node of completion of the remote update. Then,operation returns to block 3116 to determine whether the programmingidiom accelerator receives processor instruction size information fromthe virtualization layer.

Thus, the illustrative embodiments solve the disadvantages of the priorart by providing a wake-and-go mechanism for a microprocessor. When athread is waiting for an event, rather than performing a series ofget-and-compare sequences, the thread updates a wake-and-go array with atarget address associated with the event. The target address may pointto a memory location at which the thread is waiting for a value to bewritten. The thread may update the wake-and-go array using a processorinstruction within the program, a call to the operating system, or acall to a background sleeper thread, for example. The thread then goesto sleep until the event occurs.

The wake-and-go array may be a content addressable memory (CAM). When atransaction appears on the symmetric multiprocessing (SMP) fabric thatmodifies the value at a target address in the CAM, which is referred toas a “kill,” the CAM returns a list of storage addresses at which thetarget address is stored. The operating system or a background sleeperthread associates these storage addresses with the threads waiting foran even at the target addresses, and may wake the one or more threadswaiting for the event.

In one illustrative embodiment, a programming idiom acceleratoridentifies a remote update programming idiom. The programming idiomaccelerator sends the remote update to a remote node to perform anoperation on data at the remote node. A programming idiom accelerator atthe remote node receives the remote update and performs the update as alocal operation. The programming idiom accelerator at the remote nodemay request processing resources from a virtualization layer to performthe update. The programming idiom accelerator may determine a remoteupdate threshold that limits the instruction size of remote updates thatmay be sent to a remote node. The programming idiom accelerator may alsodetermine a dedicated threshold that limits the instruction size ofremote updates that may be performed by the programming idiomaccelerator.

It should be appreciated that the illustrative embodiments may take theform of a specialized hardware embodiment, a software embodiment that isexecuted on a computer system having general processing hardware, or anembodiment containing both specialized hardware and software elementsthat are executed on a computer system having general processinghardware. In one exemplary embodiment, the mechanisms of theillustrative embodiments are implemented in a software product, whichmay include but is not limited to firmware, resident software,microcode, etc.

Furthermore, the illustrative embodiments may take the form of acomputer program product accessible from a computer-usable orcomputer-readable medium providing program code for use by or inconnection with a computer or any instruction execution system. For thepurposes of this description, a computer-usable or computer-readablemedium can be any apparatus that can contain, store, communicate,propagate, or transport the program for use by or in connection with theinstruction execution system, apparatus, or device.

The medium may be an electronic, magnetic, optical, electromagnetic, orsemiconductor system, apparatus, or device. Examples of acomputer-readable medium include a semiconductor or solid state memory,magnetic tape, a removable computer diskette, a random access memory(RAM), a read-only memory (ROM), a rigid magnetic disk, and an opticaldisk. Current examples of optical disks include compact disk-read-onlymemory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.

The program code of the computer program product may compriseinstructions that are stored in a computer readable storage medium in aclient or server data processing system. In a client data processingsystem embodiment, the instructions may have been downloaded over anetwork from one or more remote data processing systems, such as aserver data processing system, a client data processing system, or aplurality of client data processing systems using a peer-to-peercommunication methodology. In a server data processing systemembodiment, the instructions may be configured for download, or actuallydownloaded, over a network to a remote data processing system, e.g., aclient data processing system, for use in a computer readable storagemedium with the remote data processing system.

A data processing system suitable for storing and/or executing programcode will include at least one processor coupled directly or indirectlyto memory elements through a system bus. The memory elements can includelocal memory employed during actual execution of the program code, bulkstorage, and cache memories which provide temporary storage of at leastsome program code in order to reduce the number of times code must beretrieved from bulk storage during execution.

Input/output or I/O devices (including but not limited to keyboards,displays, pointing devices, etc.) can be coupled to the system eitherdirectly or through intervening I/O controllers. Network adapters mayalso be coupled to the system to enable the data processing system tobecome coupled to other data processing systems or remote printers orstorage devices through intervening private or public networks. Modems,cable modems and Ethernet cards are just a few of the currentlyavailable types of network adapters.

The description of the present invention has been presented for purposesof illustration and description, and is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the art. Theembodiment was chosen and described in order to best explain theprinciples of the invention, the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A method, in a data processing system, forperforming a remote update, the method comprising: receiving, at aremote update programming idiom accelerator within the data processingsystem, a complex remote update programming idiom from a remote computersystem, wherein the complex remote update programming idiom is asequence of instructions from a process executing on the remote computersystem including a read operation for reading data from a storagelocation local to the data processing system and not local to the remotecomputer system, at least one update operation for performing anoperation on the data to form result data, and a write operation forwriting the result data to the storage location local to the dataprocessing system; determining whether the sequence of instructions islonger than a dedicated processor threshold; responsive to adetermination that the sequence of instructions is longer than thededicated processor threshold, requesting processing resources from avirtualization layer in the data processing system; receiving anallocation of processing resources from the virtualization layer;reading, by the allocated processing resources, the data from thestorage location local to the data processing system; executing, by theallocated processing resources, the sequence of instructions to performthe update operation on the data to form the result data; writing, bythe processing resources, the result data to the storage location localto the data processing system; and returning a completion notificationfrom the remote update programming idiom accelerator to the remotecomputer system informing the remote computer system that processing ofthe complex remote update programming idiom has completed on the dataprocessing system.
 2. The method of claim 1, further comprising:responsive to a determination that the sequence of instructions is notlonger than the dedicated processor threshold, performing the complexremote update by the remote update programming idiom accelerator.
 3. Themethod of claim 2, wherein performing the complex remote updatecomprises: reading, by the remote update programming idiom accelerator,the data from the storage location local to the data processing system;executing, by the remote update programming idiom accelerator, thesequence of instructions to perform the update operation on the data toform the result data; writing, by the remote update programming idiomaccelerator, the result data to the storage location local to the dataprocessing system; and returning a completion notification from theremote update programming idiom accelerator to the remote node informingthe remote node that processing of the complex remote update programmingidiom has completed on the remote update programming idiom accelerator.4. The method of claim 1, further comprising: dynamically determining,by the remote update programming idiom accelerator, the dedicatedprocessor threshold based on remote update demand.
 5. The method ofclaim 1, further comprising: determining an instruction size thresholdvalue for the data processing system; and sending the instruction sizethreshold value for the data processing system to each remote computersystem connected to the data processing system within a heartbeatmessage, wherein each remote computer system determines whether to senda remote update programming idiom to the data processing system based ona comparison of the remote update programming idiom to the instructionsize threshold.
 6. The method of claim 5, wherein determining theinstruction size threshold comprises: receiving processor utilizationinformation from the virtualization layer; and determining theinstruction size threshold based on the processor utilizationinformation.
 7. The method of claim 1, further comprising: detecting, bythe remote update programming idiom accelerator, a second complex remoteupdate in an instruction sequence of a thread running on a processingunit within the data processing system, wherein the second complexremote update programming idiom includes a second read operation forreading second data from a storage location at a second remote computersystem, a second sequence of instructions for performing a second updateoperation on the second data to form second result data, and a secondwrite operation for writing the second result data to the storagelocation at the second remote computer system; determining whether thesecond sequence of instructions is longer than an instruction sizethreshold associated with the second remote computer system; andresponsive to a determination that the second sequence of instructionsis not longer than the instruction size threshold associated with thesecond remote computer system, transmitting the second complex remoteupdate programming idiom from the remote update programming idiomaccelerator to the second remote computer system to perform the secondupdate operation on the second data at the second remote computersystem.
 8. The method of claim 7, further comprising: responsive to adetermination that the second sequence of instructions is longer thanthe instruction size threshold associated with the second remotecomputer system, performing the second complex remote update by thethread running on the processing unit by creating a communicationssession between the thread and the second remote computer system.
 9. Themethod of claim 7, further comprising: receiving the instruction sizethreshold associated with the second remote computer system within aheartbeat message from the second remote computer system.
 10. The methodof claim 1, wherein the remote update programming idiom acceleratorcomprises a dedicated processor associated with a bus within the dataprocessing system.
 11. A data processing system, comprising: at leastone processing unit implemented in hardware; a virtualization layer; anda remote update programming idiom accelerator, wherein the remote updateprogramming idiom accelerator is configured to receive a complex remoteupdate programming idiom from a remote computer system, wherein thecomplex remote update programming idiom is a sequence of instructionsfrom a process executing on the remote computer system including a readoperation for reading data from a storage location local to the dataprocessing system and not local to the remote computer system, at leastone update operation for performing an operation on the data to formresult data, and a write operation for writing the result data to thestorage location local to the data processing system; wherein the remoteupdate programming idiom accelerator is configured to determine whetherthe sequence of instructions is longer than a dedicated processorthreshold, responsive to a determination that the sequence ofinstructions is longer than the dedicated processor threshold, request aprocessing unit from the virtualization layer in the data processingsystem, and receive an allocation of a processing unit from thevirtualization layer; wherein the allocated processing unit isconfigured to read the data from the storage location local to the dataprocessing system, execute the sequence of instructions to perform theupdate operation on the data to form result data, and write the resultdata to the storage location local to the data processing system; andwherein the remote update programming idiom accelerator is configured toreturn a completion notification from the remote update programmingidiom accelerator to the remote computer system informing the remotecomputer system that processing of the complex remote update programmingidiom has completed on the data processing system.
 12. The dataprocessing system of claim 11, wherein the remote update programmingidiom accelerator is further configured to, responsive to adetermination that the sequence of instructions is not longer than thededicated processor threshold, perform the complex remote update by theremote update programming idiom accelerator.
 13. The data processingsystem of claim 12, wherein performing the complex remote updatecomprises: reading, by the remote update programming idiom accelerator,the data from the storage location local to the data processing system;executing, by the remote update programming idiom accelerator, thesequence of instructions to perform the update operation on the data toform the result data; writing, by the remote update programming idiomaccelerator, the result data to the storage location local to the dataprocessing system; and returning a completion notification from theremote update programming idiom accelerator to the remote computersystem informing the remote computer system that processing of thecomplex remote update programming idiom has completed on the remoteupdate programming idiom accelerator.
 14. The data processing system ofclaim 11, wherein the remote update programming idiom accelerator isfurther configured to dynamically determine the dedicated processorthreshold based on remote update demand.
 15. The data processing systemof claim 11, wherein the remote update programming idiom accelerator isfurther configured to: determine an instruction size threshold value forthe data processing system; and send the instruction size thresholdvalue for the data processing system to each remote computer systemconnected to the data processing system within a heartbeat message,wherein each remote computer system determines whether to send a remoteupdate programming idiom to the data processing system based on acomparison of the remote update programming idiom to the instructionsize threshold.
 16. The data processing system of claim 15, whereindetermining the instruction size threshold comprises: receivingprocessor utilization information from the virtualization layer; anddetermining the instruction size threshold based on the processorutilization information.
 17. The data processing system of claim 11,wherein the remote update programming idiom accelerator is furtherconfigured to: detect a second complex remote update in an instructionsequence of a thread running on a processing unit within the dataprocessing system, wherein the second complex remote update programmingidiom includes a second read operation for reading second data from astorage location at a second remote computer system, a second sequenceof instructions for performing a second update operation on the seconddata to form second result data, and a second write operation forwriting the second result data to the storage location at the secondremote computer system; determine whether the second sequence ofinstructions is longer than an instruction size threshold associatedwith the second remote computer system; and responsive to adetermination that the second sequence of instructions is not longerthan the instruction size threshold associated with the second remotecomputer system, transmit the second complex remote update programmingidiom from the remote update programming idiom accelerator to the secondremote computer system to perform the second update operation on thesecond data at the second remote computer system.
 18. The dataprocessing system of claim 17, wherein the remote update programmingidiom accelerator is further configured to: responsive to adetermination that the second sequence of instructions is longer thanthe instruction size threshold associated with the second remotecomputer system, perform the second complex remote update by the threadrunning on the processing unit by creating a communications sessionbetween the thread and the second remote computer system.
 19. The dataprocessing system of claim 17, wherein the remote update programmingidiom accelerator is further configured to: receive the instruction sizethreshold associated with the second remote computer system within aheartbeat message from the second remote computer system.
 20. The dataprocessing system of claim 11, wherein the remote update programmingidiom accelerator comprises a dedicated processor associated with a buswithin the data processing system.